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ABSTRACT 


The  activities  and  tasks  performed  by  firefighters  when  responding  to  emergencies 
caused  by  asymmetric  threats  to  homeland  security  mirror  the  activities  and  tasks  that 
firefighters  regularly  employ  when  responding  to  conventional  emergencies.  However, 
the  learning  opportunities  created  by  conventional  incidents  are  not  routinely  exploited 
for  the  purposes  of  preparing  firefighters  to  respond  to  incidents  of  asymmetric  origin. 
Instead,  homeland  security  training  and  education  is  often  conducted  in  a  manner  that  is 
stand-alone  and  requires  a  dedicated  budget. 

The  policy  analysis  conducted  assesses  the  similarities  and  differences  between 
mitigation  procedures  and  technical  skills  used  when  responding  to  incidents  of  similar 
nature  but  different  origin  and  whether  or  not  formal  critiques  and  after  action  reports 
from  conventional  incidents  can  be  used  to  effectively  support  the  long-term  sustainment 
of  specialized  training  and  education.  Efficiency,  process  values,  and  robustness  and 
unprovability  are  the  criterion  used  to  conduct  a  modified  cost-benefit  analysis.  The 
findings  suggest  that  expanding  the  scope  of  formal  critiques  and  after  action  reports 
from  conventional  incidents  to  include  “what  if’  questions  about  potential  incidents  of 
asymmetric  origin  does  facilitate  the  long-term  sustainment  of  specialized  training  and 
education  programs  in  a  manner  that  capitalizes  on  adult  and  organizational  learning 
theory  principles. 
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I.  INTRODUCTION 


A.  PROBLEM  STATEMENT 

The  responsibilities  of  the  fire  service  related  to  the  asymmetrical  threats  to 
homeland  security  fall  primarily  within  the  response  mission  area  of  the  emergency 
management  cycle  and  focus  on  consequence  management.1  The  asymmetrical  threats 
for  which  the  fire  service  must  be  prepared  to  respond  include  emergencies  caused  by 
weapons  of  mass  destruction  (WMD),  such  as  weapons  comprised  of  chemical, 
biological,  radiological,  and  nuclear  materials  and/or  improvised  explosive  devices. 
Consistent  with  traditional  fire  service  priorities,  consequence  management  for  WMD 
incidents  focuses  on  life  safety,  property  preservation,  and  protection  of  the  environment. 

Many  of  the  activities  and  tasks  performed  by  the  fire  service  during  the  response 
to  emergencies  caused  by  a  WMD  incident  mirror  the  activities  and  tasks  that  firefighters 
routinely  employ  when  responding  to  emergencies  of  conventional  origin2.  However, 
specialized  training  and  education  programs  that  focus  on  homeland  security  issues  are 
commonly  conducted  in  a  stand-alone  manner  and  require  a  dedicated  budget. 
Furthermore,  the  National  Preparedness  System  promotes  the  evaluation  of  capability 
levels  for  responding  to  homeland  security  incidents  through  staged  exercises  without 
assessing  responder  performance  during  the  mitigation  of  real-world  incidents  that 
require  the  use  of  comparable  skill  sets. 

Sustainment  of  specialized  training  and  education  programs  is  often  dependent 
upon  the  availability  of  grant  funding.  Since  2002,  the  fire  service  is  one  of  many 
disciplines  fortunate  to  receive  federal  funding  dedicated  to  supporting  specialized 
homeland  security  training  and  education  programs.  The  costs  associated  with  these 

1  Recent  efforts  to  implement  the  Fire  Service  Intelligence  Enterprise  are  expanding  the  scope  of  fire 
service  responsibilities  to  include  prevention  activities,  but  overall  prevention  activities  remain  in  the  law 
enforcement  arena  and  are  not  addressed  in  this  paper. 

2  The  author  acknowledges  that  a  situational  assessment  of  emergencies  scenes  caused  by  asymmetric 
threats  and  those  with  origins  unrelated  to  homeland  security  will  yield  some  difference  among  variables 
that  must  be  considered  prior  to  making  tactical  decisions.  Recommendations  for  incorporating  these 
differences  into  training  and  education  initiatives  will  be  discussed  in  Chapter  VI. 
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programs  are  allowable  homeland  security  grant  expenditures  (Federal  Emergency 
Management  Agency,  2009).  While  this  funding  enables  many  fire  departments  to 
enhance  capabilities,  the  human  resources  and  time  required  to  conduct  these  specialized 
programs  are  often  limited  and  compete  with  resources  required  for  other  fire  service 
responsibilities.  As  fire  department  corporate  budgets  continue  to  become  leaner,  the 
sustainment  of  existing  specialized  training  and  education  programs  become  increasingly 
contingent  upon  the  receipt  of  federal  grant  funding. 

In  comparison  to  the  fire  service’s  historical  operational  focus,  the  asymmetric 
threats  to  homeland  security  are  relatively  new.  These  current  day  threats  are  unlikely  to 
disappear  in  the  years  to  come.  The  influx  of  funding  post-  9/1 1  created  a  unique 
opportunity  for  the  fire  service  to  procure  specialized  equipment  and  train  personnel  for 
responding  to  these  new  challenges.  However,  as  the  novelty  of  terrorism  continues  to 
wear  off  and  the  threats  become  more  common  place,  the  challenge  of  preventing 
preparedness  and  response  complacency  among  first  responders  increases.  Furthennore, 
if  future  homeland  security  grant  programs  are  not  maintained  at  the  current  funding 
level,  it  is  unlikely  that  specialized  training  and  education  programs  can  be  maintained  at 
the  current  levels. 

If  specialized  training  and  education  programs  are  not  sustained,  the  fire  service 
faces  the  risk  of  responders  becoming  complacent.  The  consequences  of  responder 
complacency  may  include  avoidable  death  and/or  injury  to  first  responders  and  members 
of  the  public.  Additionally,  excessive  damage  to  property  and  the  environment  may 
occur  if  proper  mitigation  strategies  are  not  employed  in  a  timely  manner.  In  order  to 
decrease  the  risk  of  responder  complacency,  the  fire  service  must  integrate  cost-efficient 
strategies  for  incorporating  specialized  training  and  education  programs  into  routine 
operations  and  training  plans. 

B.  RESEARCH  QUESTIONS 

The  two  research  questions  that  this  thesis  seeks  to  answer  are: 

•  What  are  the  similarities  and  differences  between  capabilities  required  to 
mitigate  emergencies  caused  by  conventional  and  asymmetric  threats  to  public 
safety? 
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•  How  can  local  fire  departments  use  formal  critiques  and  after  action  reports 
(AARs)  to  sustain  specialized  training  and  education  for  responding  to 
emergencies  caused  by  asymmetric  threats? 

C.  RESEARCH  ARGUMENT 

Many  of  the  activities  and  tasks  identified  in  the  Target  Capabilities  List  (TCL) 
for  mitigating  emergencies  caused  by  asymmetric  threats  to  homeland  security  mirror  the 
activities  and  tasks  routinely  employed  by  firefighters  to  mitigate  emergencies  of 
conventional  origin.  However,  training  and  education  that  focus  on  responding  to 
emergencies  caused  by  asymmetric  threats  are  commonly  conducted  in  a  stand-alone 
manner  and  require  a  dedicated  budget.  Sustainment  of  specialized  training  and 
education  is  often  contingent  upon  the  availability  of  grant  funding. 

Since  the  year  2002,  specialized  training  and  education  programs  within  the  fire 
service  have  largely  focused  on  developing  and  enhancing  the  capabilities  required  to 
mitigate  incidents  caused  by  asymmetric  threats  to  homeland  security.  Fortunately,  state 
and  local  jurisdictions  have  received  federal  funding  to  support  these  initiatives. 
However,  the  continuation  of  federal  funding  for  these  programs  at  the  current  level  is 
not  guaranteed,  although  the  asymmetric  threats  to  homeland  security  that  we  currently 
face  are  not  likely  to  be  diminished.  In  order  to  sustain  preparedness  capabilities  for 
years  to  come,  the  fire  service  must  explore  possibilities  for  maximizing  training  and 
education  opportunities  that  come  with  a  price  tag  that  is  cheaper  than  the  costs 
associated  with  didactic  training  and  full-scale  exercises,  but  which  provide  similar 
learning  opportunities. 

There  are  a  number  of  similarities  between  the  capabilities  required  to  mitigate 
emergencies  caused  by  conventional  threats  to  public  safety  and  asymmetric  threats  to 
homeland  security.  Exploitation  of  these  similarities  can  help  a  fire  department  to  sustain 
the  capabilities  for  mitigating  homeland  security  incidents.  By  incorporating  discussions 
on  these  similarities  into  formal  critiques  and  AARs  from  real-world  incidents,  fire 
departments  create  a  cost-efficient  mechanism  for  increasing  the  frequency  in  which 
firefighters  actively  engage  in  preparing  to  mitigate  incidents  caused  by  asymmetric 
threats. 
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The  infrequent  occurrence  of  homeland  security  incidents  within  the  United  States 
limits  the  opportunities  for  firefighters  to  employ  the  specialized  capabilities  developed 
and/or  enhanced  since  2001  during  the  response  to  incidents  of  asymmetric  origin. 
However,  urban  fire  departments  routinely  employ  many  of  the  homeland  security  target 
capabilities  when  responding  to  incidents  of  conventional  origin.  For  example, 
hazardous  material  response,  decontamination,  and  search  and  rescue  capabilities  are 
used  during  incidents  involving  industrial  chemical  accidents  and  collapsed  or 
structurally  unstable  buildings.  The  manner  of  execution  for  the  core  activities  and  tasks 
required  for  successful  incident  mitigation  as  defined  in  the  TCL  remains  consistent 
across  emergencies  that  originate  from  conventional  and  asymmetric  causes.  For 
example,  establishment  of  an  incident  command  post  in  a  controlled  area  outside  of  the 
hot  zone  is  required  for  structural  fires,  hazardous  materials  incidents,  and  emergencies 
that  result  from  the  detonation  of  an  improvised  explosive  device.  Confirmation  that  the 
activities  and  tasks  employed  to  mitigate  emergencies  caused  by  conventional  and 
asymmetric  threats  are  indeed  comparable  is  required. 

Perfonnance  of  the  activities  and  tasks  included  on  the  TCL  during  the  response 
to  incidents  that  originate  from  conventional  causes  is  found  in  standard  incident 
documentation  completed  by  members  of  a  fire  department.  Company  officers  and  fire 
chiefs  are  required  to  document  actions  taken  to  mitigate  an  incident  by  creating  a  report 
in  the  federally  mandated  and  standardized  National  Fire  Incident  Reporting  System 
(NFIRS).  A  NFIRS  report  provides  an  incident  summary,  including  the  responding 
companies,  mitigation  strategy,  and  sequence  of  events.  Additional  incident 
documentation  is  completed  per  the  requirements  of  each  individual  fire  department’s 
documentation  policies.  Chief-level  documentation  routinely  includes  specific 
equipment  used  and  tactical  procedures  employed.  At  the  company  officer  level, 
documentation  commonly  includes  a  detailed  accounting  of  the  company’s  on-scene 
activities.  Types  of  information  in  company-specific  documentation  includes  positioning 
of  the  apparatus,  company  assignment,  sub-assignments  of  each  company  member, 
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mitigation  activities  completed,  equipment  utilized,  victims  rescued  and  treatment 
provided,  issues  encountered,  and  information  reported  to  on-scene  supervisors  and 
command  personnel. 

If  the  activities  and  tasks  employed  during  the  mitigation  of  incidents  caused  by 
conventional  and  asymmetric  threats  are  comparable,  then  using  formal  critiques  and 
AARs  from  real-world  incidents  can  help  to  sustain  specialized  training  and  education  for 
homeland  security  preparedness.  Integration  of  “what  if’  questions  regarding  homeland 
security-specific  variables  during  formal  critiques  of  real-world  incidents  create  cost- 
efficient  learning  opportunities.  Integration  of  such  questions  provides  opportunities  for 
firefighters  to  immediately  analyze  and  apply  theoretical  information  to  what  was  learned 
during  the  critique  discussion.  The  constructs  of  adult  learning  theory  and  higher-order 
learning  support  this  proposed  strategy.  This  andragogical  learning  model  provides  both 
individual  and  organizational  learning  opportunities.  The  incorporation  of  the  higher 
order  learning  construct  of  experimentation  into  the  formal  critique  process  supports  the 
recommendation  of  Bazennan  (2006)  and  Henning  (2009)  to  consider  multiple  causes  of 
the  problem  before  deciding  upon  surprise  avoidance  tactics. 

Formal  critiques,  the  purpose  of  which  is  to  critically  analyze  what  worked  well 
during  the  response,  what  did  not  work  so  well,  and  what  can  be  done  to  improve  future 
responses  to  similar  types  of  incidents,  are  not  difficult  to  conduct  or  document  and  are  a 
recommended  best  practice  within  the  fire  service  (Ockershausen,  2008).  The  conduct  of 
formal  critiques  and  development  of  AARs  are  not  contingent  upon  the  availability  of 
specialized  funding. 

Formal  critiques  require  the  participation  of  all  key  personnel  involved  in  the 
incident  response.  Within  the  fire  service,  this  means  representation  from  most,  if  not  all, 
responding  companies,  chief  officers,  and  at  times,  external  agencies.  To  ensure  an 
effective  process  and  optimal  participation,  department  leadership  must  prioritize 
opportunities  for  learning  and  perfonnance  improvement.  This  may  require  temporarily 
suspending  some  companies’  availability  for  assignment  to  emergency  calls  received 
during  the  critique.  Critiques  that  critically  evaluate  the  performance  of  both  the 
company  and  command  levels  facilitate  experiential  learning. 
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Formal  critique  participants  often  possess  different  levels  of  experience  and 
subject  matter  knowledge.  A  skilled  critique  facilitator  can  take  advantage  of 
opportunities  to  educate  those  in  the  group  who  are  less  knowledgeable  on  technical 
points  that  are  critical  to  a  successful  response,  such  as  ensuring  proper  strut  placement 
during  a  trench  rescue  prior  to  responders  entering  the  space.  Although  a  formal  critique 
centers  on  a  historical  event,  it  provides  a  dynamic  opportunity  to  train  for  the  future. 

Situation-dependant  variables  prevent  two  emergencies  from  being  exactly  the 
same.  However,  these  variables  rarely  change  the  activities  and  tasks  required  for 
incident  mitigation.  Instead,  these  variables  provide  different  factors  to  consider  when 
devising  mitigation  strategies.  For  example,  a  command  post  is  always  established 
during  a  hazardous  materials  incident.  However,  the  location  of  the  command  post  is 
determined  based  on  the  chemical(s)  of  concern  and  the  direction  of  the  wind. 
Firefighters,  company  officers,  and  fire  chiefs  must  be  skilled  at  recognizing  situation- 
dependant  variables  and  adjusting  mitigation  strategies  accordingly.  Formal  critiques 
provide  opportunities  for  the  participants  to  consider  how  hypothetical  variables  may 
affect  the  mitigation  strategy  for  the  incident  being  critiqued. 

By  introducing  “what  if’  questions  during  the  critique  process,  the  facilitator 
challenges  the  group  to  apply  the  lessons  learned  from  the  analysis  of  the  recent  response 
to  a  potential  future  situation  that  is  of  similar  nature  but  occurred  under  different 
circumstances.  The  facilitator  is  positioned  to  teach  the  group  about  specific  variables 
that  must  be  accounted  for  when  responding  to  incidents  caused  by  asymmetric  threats  to 
homeland  security.  The  structured  but  open  dialog  format  of  a  critique  provides  a  venue 
for  group  members  to  experiment  with  potential  decisions  by  discussing  how  each  option 
or  consideration  may  impact  operations.  This  process  facilitates  learning  from  each  other 
through  the  exchange  of  information  and  drawing  on  the  collective  experience  of  the 
group  without  incurring  expenses  associated  with  a  stand-alone  training  or  preparedness 
exercise. 

The  establishment  of  a  balance  among  the  demands  of  administrative,  training, 
emergency  response  and  equipment  maintenance  responsibilities  of  a  fire  department  is  a 
difficult  challenge.  The  primary  responsibility  of  a  fire  department  is  responding  to  9-1-1 
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calls  in  order  to  protect  the  health  and  safety  of  a  jurisdiction’s  population.  This 
responsibility,  coupled  with  emergency  response  often  being  the  true  passion  of  those 
who  occupy  administrative  command  positions  within  a  fire  department,  makes  it  easy 
for  the  response  function  to  be  prioritized  at  the  expense  of  others.  However,  if  this 
function  is  not  balanced  with  training  and  organizational  learning,  the  ability  of  a 
department  to  truly  learn  from  their  experiences  and  enhance  capabilities  is  limited.  By 
integrating  “what  if’  questions  into  fonnal  critiques  of  real-world  incident,  department 
leaders  achieve  both  emergency  response  and  homeland  security  training  and  education 
objectives. 

The  AAR  is  used  to  capture  the  discussion  of  both  the  analysis  of  the  incident 
being  critiqued  and  the  hypothetical  homeland  security  situation  discussed.  By 
incorporating  the  learning  points  from  the  critique  into  the  AAR,  a  fire  department 
provides  vicarious  learning  opportunities  to  firefighters  who  are  not  part  of  the  formal 
critique  discussion.  Written  AARs  are  conducive  to  wide  distribution  and  incur  minimal 
expense  for  use  as  a  training  tool.  Therefore,  once  a  fire  department  commits  to  the 
formal  critique  and  AAR  processes,  this  training  strategy  becomes  sustainable  since  the 
financial  resources  required  are  minimal  when  compared  to  costs  associated  with  current- 
day  specialized  training  and  education  programs. 

An  environment  that  cultivates  learning  opportunities  and  embraces  change  helps 
an  organization  to  break  the  perpetual  cycle  of  repeating  the  same  mistakes.  If 
department  leadership  is  unwilling  to  accept  constructive  criticism  and  take  action  to 
address  identified  areas  for  improvement,  the  value  of  investing  time  and  resources  in 
conducting  and  documenting  formal  critiques  is  questionable.  Such  a  department  is 
likely  to  experience  predictable  surprises  in  the  future. 

Formal  critiques  that  include  “what  if’  discussions  provide  a  cost-efficient  and 
sustainable  mechanism  for  educating  first  responders  on  incidents  caused  by  asymmetric 
threats  to  homeland  security.  The  introduction  of  new  scenarios  shortly  after  critique 
participants  have  evaluated  the  strengths  and  weaknesses  of  their  performance  to  a 
similar  situation  provides  an  opportunity  for  the  group  members  to  solidify  the  content 
learned  through  immediate  application,  a  core  principle  of  adult  learning  theory 
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(Knowles,  1989).  A  responder’s  bank  of  experiences  upon  which  to  draw  when 
confronted  with  a  unique  situation  is  expanded  by  comparing  and  contrasting  the 
similarities  and  differences  of  incidents  that  have  different  origins.  As  theorized  in  the 
Bayesian  theory  of  surprise,  “surprise  can  only  exist  in  the  presence  of  uncertainty”  (Itti 
&  Baldi,  2005,  p.  2).  If  responders  are  able  to  draw  upon  their  experiences  to  identify 
parallels  between  responses  operations  to  emergencies  caused  by  conventional  and 
asymmetric  threats,  the  probability  of  surprise  decreases  and  the  probability  of  effectively 
executing  mitigation  strategies  increases. 
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II.  LITERATURE  REVIEW 


This  literature  review  covers  four  topic  areas  that  help  to  answer  the  two  research 
questions  posed  in  this  thesis.  The  first  two  topics,  adult  learning  theory  and 
organizational  learning  theory,  provide  insight  into  the  theoretical  perspectives  that  are 
used  to  guide  the  development  of  training  and  education  programs.  Infonnation  on  the 
strengths  and  weaknesses  of  the  different  training  and  education  methods  establish 
whether  or  not  formal  critiques  and  AARs  are  appropriate  for  consideration  as  strategies 
to  help  sustain  specialized  training  and  education  programs.  The  third  topic,  surprise, 
addresses  conceptual  theories  of  surprise  that  relate  to  first  responder  preparedness  for 
mitigating  emergencies  caused  by  asymmetric  threats  to  homeland  security.  This 
discussion  explores  the  assertion  that  exploiting  commonalities  between  the  capabilities 
required  to  mitigate  emergencies  caused  by  conventional  and  asymmetric  threats  helps  to 
decrease  the  probability  that  firefighters  will  experience  surprise  when  responding  to 
homeland  security  incidents.  The  fourth  topic,  incident  critiques,  helps  to  establish  a 
baseline  understanding  of  incident  critiques  and  how  they  are  used  by  public  safety 
organizations.  This  section  shows  how  fonnal  critiques  and  AARs  utilize  the  principles 
of  adult  and  organizational  learning  theories  and  identifies  the  current  gaps  in  the 
literature  in  regards  to  using  formal  critiques  and  AARs  from  real-world  incidents  as  a 
mechanism  to  sustain  homeland  security  preparedness  capabilities. 

A.  ADULT  LEARNING  THEORY 

There  are  two  theories  of  learning  that  should  be  considered  when  contemplating 
adult  education:  reinforcement  theory  and  andragogy  (Kelly,  2006). 

B.F.  Skinner,  the  behavioral  psychologist  who  discovered  operant  conditioning 
and  articulated  the  term  “reinforcement”  as  a  scientific  principle  of  behavior,  claims  that 
reinforcements  increase  the  probability  that  a  certain  behavior  will  occur  (Epstein  & 
Skinner,  1980).  In  their  paper  Pavlov  and  Skinner:  Two  Lives  in  Science,  Catania  and 
Laties  (1999)  discuss  how  Skinner  built  upon  Russian  psychologist  Ivan  Pavolv’s  Nobel 
Prize  winning  work  on  classical  conditioning  to  continue  the  scientific  study  of  the 
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behavior  of  individuals,  which  eventually  led  to  the  development  of  reinforcement 
theory.  Reinforcement  theory  asserts  that  rewards  for  correct  behavior,  called  positive 
reinforcement,  and  punishment  for  inappropriate  behavior,  negative  reinforcement,  can 
help  an  individual  to  learn  (Kelly,  2006).  However,  in  a  paper  co-authored  with  his 
graduate  student  Robert  Epstein  titled  Resurgence  of  Responding  after  the  Cessation  of 
Response-Independent  Reinforcement,  Epstein  and  Skinner  (1980)  question  the 
probability  of  behavior  occurrences  decreasing  once  rewards  are  no  longer  provided; 
questioning  whether  or  not  true  learning  occurred.  Furthennore,  Epstein’s  (1991)  paper 
Skinner,  Creativity,  and  the  Problem  of  Spontaneous  Behavior  questions  the  impetus 
behind  the  initial  behavior  that  was  changed  through  reinforcement.  Without  fully 
understanding  the  behavior  and  the  logic  for  the  behavior,  how  can  the  determination  be 
made  that  the  behavior  change  is  directly  related  to  the  reinforcement?  Reinforcement 
theory  may  be  applicable  to  organizational  learning  during  the  early  phases  of 
transfonnational  learning  in  order  to  initiate  momentum  for  change.  However,  this 
theory  should  likely  not  be  used  in  isolation  if  sustained  behavioral  change  is  desired  or 
an  organization  wishes  to  promote  the  development  of  critical  analysis  and  decision¬ 
making  skills. 

Alexander  Kapp  first  introduced  the  concept  of  andragogy  in  his  1833  book 
Planton’s  Erziehungslehre  (Plato’s  Educational  Ideas)  (Peterson,  2009).  Andragogy  is 
learner  focused  education  that  encourages  the  adult  learner  to  move  beyond  dependency 
to  self-directed  learning.  Reg  Revans,  the  pioneer  of  the  action  learning  theory,  used  the 
principles  of  andragogy  to  support  the  notion  that  people  can  use  their  daily  jobs  as  the 
basis  for  learning.  Revans  challenged  the  conventional  model  of  learning — learn  first, 
apply  knowledge  second.  In  his  seminal  work,  Revans  successfully  demonstrated  that 
adults  who  exchange  information  can  use  the  infonnation  learned  from  each  other’s 
experiences  to  make  changes  within  their  own  organizations  and/or  individual  practices 
(Margerison,  n.d.). 

Malcolm  Knowles,  considered  the  founder  of  adult  education  theory,  used 
andragogy  as  the  foundation  for  the  development  of  adult  education  theory.  In  his  theory, 
Knowles  encourages  the  learner  to  draw  upon  personal  experiences  to  leam  how  to 
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perform  new  tasks  and  solve  problems.  Knowles  also  promotes  the  immediate 
application  of  new  information  learned  as  a  means  of  solidifying  the  new  concepts  and 
behaviors  learned.  However,  in  his  document  Everything  You  Wanted  to  Know  From 
Malcolm  Knowles,  Knowles  (1989)  cautions  that  learners  must  not  be  dumped  from  the 
pedagogical  learning  model  used  primarily  for  childhood  education  into  an  andragorical 
model  without  orientation  on  how  to  adapt  to  and  operate  within  a  self-directed  model. 
To  omit  the  orientation  phase  may  cause  learning  barriers.  Knowles  argues  that  by 
nature,  adults  are  self-directed  learners  and  can  effectively  take  responsibility  for  their 
own  learning. 

Revans’  action  learning  theory  and  Knowles’  adult  learning  theory  are  both  based 
on  the  principles  of  andragogy  and  share  the  same  conception  that  adults  can  effectively 
learn  from  their  own  and  other’s  experiences.  Both  Revans  and  Knowles  challenge  the 
traditional  teacher-centric  model  of  education,  instead  promoting  learner-centric 
education.  Charles  Margerison  (n.d.),  one  of  today’s  leading  researchers  on  continuing 
professional  development  systems,  continues  to  use  the  work  of  Kapp,  Revans,  and 
Knowles  by  incorporating  the  principles  of  andragogy,  action  learning  theory,  and  adult 
learning  theory  into  workplace  learning  strategies. 

When  combined,  reinforcement  theory  and  the  principles  of  andragogy  are 
training  and  education  strategies  that  introduce  and  help  learners  comprehend  how  to 
integrate  new  infonnation  into  decision-making  processes.  Traditional  training  and 
education  programs  within  the  fire  service  capitalize  on  both  reinforcement  and 
andragogy-based  theories.  Repetitious  review  of  information,  techniques,  and  procedures 
is  embraced  in  the  form  of  training  classes  and  daily  drills.  The  fire  services  uses 
company  schools  and  tabletop  exercises  as  opportunities  to  exchange  information  and 
strengthen  abilities  for  personnel  to  draw  on  one’s  bank  of  knowledge  and  past 
experiences  to  make  effective  decisions  when  confronted  with  a  unique  situation. 
Although  not  explicitly  stated  in  the  literature,  the  formal  critique  process  incorporates 
the  principles  of  reinforcement,  action  learning,  and  adult  education  theories.  However, 
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the  existing  critique  process  does  not  routinely  provide  opportunities  for  immediate 
application  of  new  infonnation  learned — one  of  tenants  of  adult  education  theory 
promoted  by  Knowles. 


B.  ORGANIZATIONAL  LEARNING  THEORY 

There  are  multiple  frameworks  within  organizational  learning  theory  that  describe 
how  organizations  learn  and  the  barriers  to  an  organization’s  ability  to  leam.  While  each 
framework  has  distinct  differences,  commonalities  are  also  seen. 

Norwegian  researcher  Bjarne  Espedal  (2008)  claims  that  a  commonly  accepted 
notion  found  among  learning  theory  frameworks  is  that  lower  and  higher  order  learning 
must  be  balanced  for  a  long-tenn  adaptive  or  learning  process  to  be  successful.  Although 
sometimes  referred  to  using  different  tenninology,  lower  order  learning  focuses  on 
improving  existing  practices.  In  the  exploitation-exploration  framework,  Stockholm 
University’s  Mikael  Hohnqvist  (2009)  defines  this  concept  as  exploitation,  or  focusing  on 
specific  competencies  in  order  to  refine  current  beliefs  and  strengthen  knowledge  in  a 
specific  area.  In  the  single  and  double-loop  learning  framework,  Canneli  and  Sheaffer 
(2008)  describe  this  concept  as  the  process  of  error  identification  and  correction,  or 
single-loop  learning.  Smith  and  Elliott  (2007),  two  leading  European  researchers  in  the 
area  of  crisis  management,  term  this  concept  first  order  learning  in  their  work  Exploring 
the  Barriers  to  Learning  from  Crisis:  Organizational  Learning  and  Crisis  Management. 
Overall,  the  literature  shows  agreement  that  lower  order  learning  aims  to  improve  upon 
the  known,  enhancing  core  competencies  but  not  expanding  upon  existing  capabilities. 

In  his  paper  titled  In  the  Pursuit  of  Understanding  How  to  Balance  Lower  and 
Higher  Order  Learning  in  Organizations,  Espedal  asserts  that  higher  order  learning 
involves  using  experience  to  “challenge  existing  perspectives,  routines,  and  practices  and 
to  develop  new  perspectives  on  the  future”  (p.  366).  This  mechanism  of  learning 
encourages  experimentation  with  the  unknown.  Hohnqvist’s  paper  Complicating  the 
Organization:  A  New  Prescription  for  the  Learning  Organization?  defines  the 
exploitation-exploration  perspective  as  exploration  of  experiences  through  creativity,  risk 
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taking,  and  experimentation.  From  the  double-loop  perspective,  in  the  paper  How 
Learning  Leadership  and  Organizational  Learning  From  Failures  Enhance  Perceived 
Organizational  Capacity  to  Adapt  to  the  Task  Environment,  Carmeli  and  Sheaffer  (2008) 
claim  that  learning  occurs  when  the  cause  and  effect  relationship  of  problems  is  studied 
so  that  changes  to  prevent  future  occurrences  or  improvements  can  be  instituted.  There  is 
agreement  within  the  literature  that  higher  order  learning,  which  involves  critical  analysis 
of  an  issue  and  requires  organizational  willingness  to  embrace  new  and  often  innovative 
ideas,  leads  to  organizational  change. 

Smith  and  Elliott  (2007)  identify  three  perspectives  on  organizational  learning  and 
crisis  management:  learning  from  crisis,  learning  for  crisis,  and  learning  as  crisis.  Of  the 
three  perspectives,  Smith  and  Elliott  assert  that  organizations  are  most  resistant  to 
learning  from  crisis.  The  availability  of  information,  scape-goating,  and  lack  of 
providing  serious  attention  to  the  lessons  learned  from  other  organizations  that  have 
experienced  a  crisis  are  identified  as  the  most  common  barriers  to  organizational  learning 
from  crisis.  Carmeli  and  Sheaffer’s  (2008)  study  supports  the  notion  that  an  organization 
can  decrease  the  probability  of  future  crisis  occurring  if  one  is  able  to  incorporate  lower 
and  higher  order  learning  to  leam  from  past  experiences,  including  failures. 

Key  factors  that  contribute  to  an  organization’s  ability  to  learn  include 
institutionalization  of  the  learning  process,  the  learning  perspective  and  role  of  leadership 
in  the  change  process,  and  the  ability  to  incorporate  and  balance  the  different  methods  of 
learning  (Smith  &  Elliott,  2007;  Carmeli  &  Sheaffer,  2008;  Hohnqvist,  2009).  Learning 
organizations  must  overcome  the  difficulties  associated  with  balancing  learning  methods 
and  guard  against  the  easy  temptation  of  adopting  and  universally  applying  a  single 
learning  method.  There  is  agreement  in  the  literature  that  the  methods  of  learning 
described  in  each  framework  must  be  balanced  in  order  for  organizational  learning  to 
occur.  The  right  method  must  be  selected  in  order  to  achieve  the  desired  outcome. 
Espedal  (2008),  Hohnqvist  (2009),  and  Carmeli  and  Sheaffer  (2008)  all  agree  that  if 
methods  are  not  balanced,  the  organization  will  inevitably  miss  valuable  learning 
opportunities. 
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Lower  order  learning  builds  upon  Skinner’s  reinforcement  theory  by  promoting 
the  continuation  of  effective  behaviors  and  identifying  behaviors  to  improve  or  correct. 
From  an  organizational  perspective,  higher  order  learning  is  similar  to  action  learning  and 
adult  education  theories  in  that  it  encourages  critical  analysis  and  the  use  of  past 
experiences  to  identify  opportunities  for  learning  and  organizational  change.  Similar  to 
adult  learning  theories,  the  literature  supports  combining  lower  order  and  higher  order 
learning  to  optimize  opportunities  for  organizational  learning. 

The  process  recommended  by  the  U.S.  Fire  Administration  for  conducting  formal 
critiques  and  developing  AARs  incorporates  the  use  of  organizational  learning  theory 
principles.  However,  as  discussed  in  the  Incident  Critiques  and  After  Action  Reports 
section  of  this  literature  review,  the  critique  process  does  not  currently  take  full 
advantage  of  the  higher  order  learning  construct  of  experimentation  to  develop  new 
perspectives  on  the  future. 

C.  SURPRISE 

The  concept  of  surprise  warrants  consideration  when  discussing  capabilities  for 
responding  to  asymmetric  threats  to  homeland  security.  Webster’s  Dictionary  defines  the 
tenn  “surprise”  as  “an  attack  made  without  warning”  (Surprise,  2009).  The  literature  on 
surprise  provides  much  deeper  insight  on  how  this  phenomenon  can  be  applied  to  both 
conventional  threats  to  public  safety  and  asymmetric  threats  to  homeland  security. 

U.S.  Army  War  College  Professor  Michael  Handel  defines  the  theory  of  surprise 
in  his  article  The  Yom  Kippur  War  and  the  Inevitability  of  Surprise  as  a  theory  that 
“possesses  strong  explanatory  power”  but  “forms  a  weak  basis  for  prediction”  (Handel, 
1977,  p.  462).  One  of  Handel’s  main  arguments  is  that  misperceptions  on  the  part  of  the 
defender  facilitate  the  occurrence  of  surprise  incidents.  Incident  deconstruction  after  an 
incident  occurrence  helps  to  re-interpret  the  information  known  pre-incident  with  the 
benefit  of  hindsight.  However,  this  process  does  not  result  in  accurate  predictions  which 
may  help  one  to  employ  surprise  avoidance  strategies.  In  his  research,  Handel  (1977) 
argues  that  predictive  abilities  are  impeded  by: 

•  The  desire  to  maintain  the  status  quo; 
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•  Noise  barriers,  such  as  an  international  background  that  is  asynchronous  with 
preparations  for  attack  and  the  enemy  positioning  forces  to  increase  the 
difficulty  of  differentiating  between  decoy  and  targets; 

•  Overestimation  of  one’s  own  capabilities  and  underestimation  of  the 
opponent’s  capabilities; 

•  Inaccurate  evaluation  of  the  enemy’s  intention;  and 

•  False  alerts  that  lessen  the  vigilance  against  attack. 

In  their  book  Predictable  Surprises:  The  Disasters  You  Should  Have  Seen 
Coming,  and  How  to  Prevent  Them  (2004),  Max  H.  Bazennan  and  Michael  D.  Watkins 
define  predictable  surprise  as  “an  event  or  set  of  events  that  take  an  individual  or  group 
by  surprise,  despite  prior  awareness  of  all  of  the  information  necessary  to  anticipate  the 
events  and  their  consequences”  (p.  1).  Bazennan  and  Watkins,  two  of  the  leading 
authorities  on  managerial  decision-making,  defend  the  position  that  predictable  surprises 
occur  because  policy-makers  knowingly  make  decisions  that  facilitate  the  incident 
occurrence.  These  researchers  propose  six  characteristics  of  predictable  surprises: 

•  Leaders  are  aware  that  a  problem  exists  and  that  the  problem  will  not  fix 
itself; 

•  An  incident  can  be  expected  if  the  problem  continues  to  worsen  over  time; 

•  Problem  mitigation  is  expensive  and  the  benefits  may  not  be  realized  for  a 
significant  amount  of  time; 

•  The  benefit  of  the  financial  investment  made  to  avert  the  predictable 
surprise  is  avoidance  of  an  uncertain,  but  likely  greater  cost; 

•  The  tendency  to  maintain  the  status  quo  often  impedes  adequate 
preparation  for  predictable  surprises;  and 

•  Special  interest  groups  that  benefit  from  the  status  quo  typically  will  work 
to  block  reform. 

The  claim  that  maintaining  the  status  quo  is  a  factor  of  surprise  is  the  main  point 
of  consistency  between  Handel’s  and  Bazennan  and  Watkins’  arguments  on  why 
decision-makers  with  all  the  necessary  data  fail  to  correctly  anticipate  future  incidents. 
The  authors  disagree  on  the  appropriate  application  of  surprise  theories  and  whether  or 
not  surprise  is  avoidable.  Handel  contributes  surprise  primarily  to  misinformation  and 
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misperception,  while  Bazennan  and  Watkins  contribute  it  to  the  results  of  conscious 
decisions.  The  historical  climate  surrounding  the  different  research  endeavors  and/or  the 
advances  in  technology  that  improved  access  to  information  between  the  research  time 
periods  may  contribute  to  the  difference  in  perspectives.  While  both  used  contemporary 
examples  at  the  time  of  writing  to  substantiate  their  assertions,  the  difference  in  Handel’s 
examples  being  traditional  military  and  Bazennan  and  Watkins’  being  asymmetric  threats 
to  homeland  security  may  also  contribute  to  the  diversity  among  findings. 

Larry  Irons,  a  sociologist  who  specializes  in  organizational  intelligence, 
organizational  learning,  and  learning  communities,  uses  the  concept  of  predictable 
surprise  to  further  the  discussion  on  organizational  learning  processes.  In  his  article 
Hurricane  Katrina  as  a  Predictable  Surprise  (2006),  Irons  claims  that  incidents  with 
vivid  outcomes  tend  to  be  taken  more  seriously.  Cultivating  an  environment  in  which 
personnel  remain  vigilant  to  the  conditions  of  predictable  surprises  becomes  an  on-going 
management  challenge.  To  engage  in  surprise  avoidance,  Irons  advocates  that  managers 
and  organizational  leaders: 

•  Define  a  clear  mission  that  facilitates  workers  to  function  optimally; 

•  Promote  professionalism; 

•  Invest  in  quality  technical  training; 

•  Cultivate  leadership  qualities  in  managers; 

•  Maintain  an  organizational  structure  that  clearly  delineates  decision¬ 
making  responsibilities  at  each  rank;  and 

•  Value  ongoing  improvements. 

Harvard  Business  School  Professor  Bazennan  recently  argued  in  a  paper  titled 
Climate  Change  as  a  Predictable  Surprise  that  focusing  on  only  one  possible  cause  of  the 
problem  is  a  common  mistake  made  by  leaders  in  relation  to  predictable  surprise 
avoidance.  Ronda  Henning,  an  information  assurances  scientist,  agrees  that  the  failure  to 
address  multiple  possibilities  negatively  impacts  efforts  to  conduct  a  comprehensive 
assessment  of  vulnerabilities  and  risks,  possibly  resulting  in  missed  opportunities  to 
employ  surprise  avoidance  strategies  (Henning,  2009).  Use  of  the  elements  of  a  learning 
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organization  identified  by  Irons  helps  an  organization  identify  opportunities  to  mitigate 
multiple  factors  that  may  contribute  to  the  occurrence  of  a  predictable  surprise,  increasing 
the  likely  effectiveness  of  surprise  avoidance  tactics. 

Laurent  Itti  and  Pierre  Baldi,  university-based  computer  scientists,  put  forth  the 
Bayesian  theory  of  surprise  to  quantify  how  the  differences  between  an  observer’s 
posterior  and  prior  distributions  of  beliefs  affects  their  reaction  to  a  stimulus  that  is 
considered  a  surprise.  Itti  and  Baldi  postulate  that  only  data  observations  that 
substantially  affect  the  observer’s  beliefs  yield  surprise.  The  Bayesian  theory  is  based  on 
two  principles:  “surprise  can  only  exist  in  the  presence  of  uncertainty,  which  can  arise 
from  intrinsic  stochasticity,  missing  information,  or  limited  computing  resources”  and 
“surprise  can  only  be  defined  in  a  relative,  subjective  manner  and  is  related  to  the 
expectations  of  the  observer”  (2005,  p.  2). 

The  principles  of  Bayesian  theory  support  Carmeli  and  Sheaffer’s  (2008) 
argument  that  learning  from  past  experiences  decreases  the  probability  of  an  organization 
experiencing  a  similar  crisis  in  the  future  since  surprise  cannot  occur  unless  a  stimulus 
causes  a  change  in  beliefs.  Learning  from  previous  incidents  that  share  common 
characteristics  with  the  incident  at  hand  can  facilitate  surprise  mitigation.  Revan’s  action 
learning  theory  and  Knowles  adult  learning  theory  support  adults  using  personnel 
experience  to  leam. 

Many  of  the  principles  of  surprise  found  in  the  literature  are  commonly  used  by 
the  fire  service.  However,  incorporation  of  these  principles  into  routine  operations  by  the 
fire  service  pre-dates  the  fonnalization  of  the  theories  of  surprise.  While  situation- 
specific  variables  of  an  emergency  are  practically  impossible  to  predict  in  advance,  the 
fire  service  has  successfully  identified  the  types  of  emergencies  and  categories  of 
potential  variables  for  which  they  must  be  prepared  to  respond.  Fire  service  personnel 
are  trained  how  to  mitigate  the  consequences  of  incidents  caused  by  these  threats, 
decreasing  the  level  of  surprise  firefighters  experience  when  confronted  with  an 
emergency.  Although  the  theories  of  surprise  may  be  relatively  new,  their  tenants  are  not 
so  to  the  fire  service. 
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D.  INCIDENT  CRITIQUES  AND  AFTER  ACTION  REPORTS 

The  Department  of  Homeland  Security  (DHS)  uses  the  abbreviation  “AAR”  to 
mean  “after  action  review”  (DHS,  2007a).  The  United  States  Army  uses  “AAR”  to  mean 
“after  action  report”  (Garvin,  2000).  Although  both  terms  are  related  since  the  discussion 
that  occurs  during  the  after  action  review  is  typically  captured  in  the  AAR,  a  distinction 
between  the  two  is  required  since  they  involve  different  processes.  For  purposes  of  this 
thesis  project,  “AAR”  is  used  for  the  “after  action  report”.  The  term  “formal  critique”  or 
“critique”  is  used  in  place  of  “after  action  review.” 

A  critique  is  a  training  strategy  used  by  many  professions  and  organizations  to 
help  those  involved  in  an  incident  or  event  leam  by  comparing  the  expected  outcome 
with  the  actions  taken  to  arrive  at  the  actual  outcome  (Garvin,  2000;  Ockershausen,  2008; 
DHS,  2007a;  Turner,  2007;  Emergency  Response  and  Crisis  Management  Technical 
Assistance  Center,  2007).  Critiques  are  recommended  for  both  the  response  to  real-world 
incidents  and  staged  preparedness  exercises.  The  findings  in  David  Garvin’s  paper  U.S. 
Army’s  After  Action  Reviews:  Seizing  the  Chance  to  Learn  agree  with  the 
recommendations  in  the  U.S.  Fire  Administration’s  Special  Report:  The  After-Action 
Critique:  Training  Through  Lessons  Learned  (Ockershausen,  2008)  on  the  four  main 
questions  around  which  a  critique  should  focus: 

•  What  did  we  set  out  to  do? 

•  What  did  we  actually  do? 

•  Of  the  actions  taken,  what  worked  or  did  not  work?  Why? 

•  What  should  we  do  next  time? 

The  first  two  questions  help  discussion  participants  to  develop  a  common 
understanding  of  the  incident  through  recreating  the  timeline.  The  last  two  questions 
require  the  participants  to  critically  analyze  the  cause  and  effect  relationship  of  each 
action.  The  U.S.  Fire  Administration,  an  agency  within  the  DHS,  supports  the  position 
that  the  analytic  process  helps  to  identify  practices  that  should  be  maintained,  training 
needs,  effectiveness  of  organizational  policies  and  standard  operating  procedures,  and 
trends  or  patterns  in  errors  during  operations  that  must  be  corrected  (Ockershausen, 
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2008).  However,  the  introduction  of  theoretical  variables  that  may  occur  during  future 
incidents  of  similar  nature  to  the  incident  being  critiqued  (a.k.a.  “what  if’  questions)  is 
not  routinely  incorporated  into  the  critique  discussion. 

The  formal  critique  methodology  of  structured  but  open  dialog  among  participants 
is  indicative  of  an  appreciative  learning  system.  Naval  Postgraduate  School  Professor 
Frank  Barrett  states  that  appreciative  learning  cultures  “accentuate  the  successes  of  the 
past,  evoke  images  of  possible  futures,  and  create  a  spirit  of  restless,  on-going  inquiry 
that  empowers  members  to  new  levels  of  activity”  (Barrett,  1995,  p.  4).  A  critique  helps 
an  organization  to  develop  the  four  competencies  that  Barrett  identifies  as  necessary  for 
organization  to  survival  and  flourish:  affirmative  competence-  focusing  on  strengths, 
successes,  and  potentials;  expansive  competence-  challenging  conventional  practices; 
generative  competence-  developing  integrative  systems  that  promote  realization  of  the 
effects  caused  by  individual  actions;  and  collaborative  competence-  facilitating  ongoing 
discussion  and  exchange  of  ideas.  Critiques  embody  many  of  Knapp’s  principles  of 
andragogy  and  concepts  from  both  Revans’  action  learning  theory  and  Knowles’  adult 
learning  theory. 

The  U.S.  Fire  Administration,  whose  mission  is  to  foster  a  solid  foundation  in 
prevention,  preparedness  and  response  by  providing  leadership  to  local  fire  and 
emergency  services  agencies,  considers  the  AAR  to  be  the  most  important  part  of  the 
critique  process  since  it  is  used  to  document  the  lessons  learned  from  an  incident 
(Ockershausen,  2008).  Although  a  standardized  template  for  AARs  for  real-world 
incidents  has  yet  to  be  created,  the  U.S.  Fire  Administration  provides  an  overview  of  the 
recommended  topic  areas  for  inclusion  in  the  written  report.  The  recommendations  are 
generic  so  that  they  can  be  applied  to  a  wide  range  of  incident  and  event  types. 

The  DHS  also  advocates  the  incorporation  of  critiques  and  AARs  into  homeland 
security  exercises.  The  Homeland  Security  Exercise  and  Evaluation  Program  (HSEEP)  is 
a  capabilities  based  program  that  provides  national  standards  for  conducting  homeland 
security  exercises.  The  Program  helps  exercise  planners  integrate  elements  of  the 
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National  Preparedness  System3  to  document  achievement  of  preparedness  capabilities  in 
a  standardized  manner  (DHS,  2007a).  The  component  of  the  National  Preparedness 
System  that  most  closely  tied  to  HSEEP  is  the  TCL.  The  TCL  provides  guidance  to  state 
and  local  government  agencies  on  the  capabilities  that  the  federal  government  deems  are 
necessary  to  respond  to  terrorist  attacks  and/or  natural  disasters  (DHS,  2007b).  The 
HSEEP  policy  and  program  documents  include  a  standard  template  for  developing  an 
AAR  and  an  improvement  plan  (IP)  and  instructions  on  how  to  use  the  documents. 
Through  use  of  the  HSEEP  AAR  and  IP,  jurisdictions  can  systematically  track  progress 
in  attaining  desired  preparedness  capabilities.  However,  given  the  commonly  accepted 
notion  of  the  difficulties  involved  in  simulating  real-world  conditions  during  an  exercise, 
one  may  question  the  extent  to  which  exercise  AARs  and  IPs  represent  true  preparedness 
capabilities  since  the  findings  are  obtained  using  simulated  conditions.  Literature  on  this 
topic  is  not  found. 

Exercise  Evaluation  Guides,  or  EEGs,  are  standardized  instruments  for  assessing 
responder  performance  during  an  exercise.  Evaluators  use  EEGs  to  document  player 
performance  in  relation  to  each  target  capability.  DHS  advocates  the  use  of  EEGs 
through  the  HSEEP  program  (DHS,  2007a).  Christine  Bradshaw  and  Thomas  Bartenfeld 
(2009),  scientists  at  the  Centers  for  Disease  Control  and  Prevention  who  are  working  to 
develop  a  national  performance  measurement  system  to  assess  all-hazards  preparedness, 
also  advocate  the  use  of  EEGs  to  measure  perfonnance  during  exercises.  Bradshaw  and 
Bartenfeld  argue  that  the  strength  of  the  EEG  lies  in  the  fact  that  the  evaluation  is 
completed  by  a  third  party  as  opposed  to  self-reported  assessments,  which  likely  yields  an 
assessment  that  is  comparatively  more  objective.  However,  the  authors  note  that  EEGs 
commonly  emphasize  evaluation  based  on  the  timeliness  of  actions.  Given  that  exercises 


3  As  a  strategy  to  organize  homeland  security  preparedness  activities,  DHS  established  the  National 
Preparedness  System.  The  goal  of  this  System  is  to  “achieve  and  sustain  coordinated  capabilities  to 
prevent,  protect  against,  respond  to,  and  recover  from  all  hazards  in  a  way  that  balances  risk  with 
resources”  (DHS,  2008).  The  six  components  of  this  System  are  the  1).  National  Preparedness  Goal;  2). 
National  Planning  Scenarios;  3).  Universal  Task  List;  4).  Target  Capabilities  List;  5).  National  Incident 
Management  System;  and  6).  National  Response  Framework. 
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are  based  on  simulated  conditions  with  artificial  timeframes  that  often  are  not  reflective 
of  real-world  conditions,  one  may  question  the  utility  of  basing  an  assessment  of 
preparedness  capabilities  on  EEG  documentation. 

The  literature  confirms  that  the  fire  service  has  endorsed  formal  critiques  and 
AARs  a  recommended  practice  for  learning  from  real-world  incidents  and  preparedness 
exercises.  However,  no  literature  is  found  on  integrating  the  elements  of  the  National 
Preparedness  System  into  critiques  and  AARs  from  the  response  to  real-world  incidents 
to  assess  and  document  preparedness  capabilities.  Literature  is  not  found  on  possibilities 
for  expanding  the  critique  discussion  to  include  theoretical  variables  for  increasing 
opportunities  to  draw  parallels  between  similar  types  of  emergencies  that  originate  from 
different  causes;  potentially  increasing  the  number  of  opportunities  for  organizational 
learning  by  fire  departments.  It  is  unknown  if  fonnal  critiques  and  AARs  from 
conventionally-caused  incidents  can  be  used  by  the  fire  service  as  a  training  tool  to  help 
avoid  surprise  during  incidents  caused  by  asymmetric  threats  to  homeland  security. 

E.  SUMMARY 

In  summary,  appreciative  learning  cultures  embrace  the  principles  of  andragogy- 
based  and  higher-order  learning  theories.  Application  of  the  concepts  contained  within 
each  of  these  theories  helps  an  organization  to  examine  their  existing  norms  in  order  to 
meet  the  challenges  that  they  face.  Learning  through  reinforcement  and  lower-order 
learning  techniques  is  beneficial.  However,  strategies  based  upon  these  theories  alone 
typically  focus  on  only  correcting  the  end  result  of  an  problem  and  do  not  strive  to 
resolve  underlying  issues,  commonly  resulting  in  reoccurrence  of  the  problem.  The 
literature  argues  the  importance  of  organizations  learning  to  embrace  multiple  learning 
theories  and  finding  a  balance  among  them  in  order  to  ensure  that  learning  opportunities 
are  not  inadvertently  missed. 

The  principles  upon  which  adult  and  organizational  learning  theories  are  based 
provide  support  for  the  U.S.  Fire  Administration’s  recommendation  that  fire  service 
organizations  use  formal  critiques  and  AARs  as  tools  for  learning.  These  tools 
incorporate  concepts  of  reinforcement,  action  learning,  adult  education,  lower  order 
learning,  and  higher  order  learning  theories. 
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The  literature  disagrees  on  whether  or  not  surprise  is  avoidable  and  the 
appropriate  application  of  surprise  theory.  The  contemplation  of  surprise  in  relation  to 
asymmetric  threats  to  homeland  security  raises  the  question  of  whether  or  not  incidents 
caused  by  asymmetric  threats  are  truly  predictable. 

Finally,  conducting  formal  critiques  and  developing  AARs  after  real-world 
emergencies  and  preparedness  exercises  is  recommended  by  the  U.S.  Fire  Administration 
and  the  DHS  HSEEP  program.  Critiques  and  AARs  help  individuals  and  organizations  to 
leam  by  critically  analyzing  the  actions  taken  to  identify  practices  that  worked  well  and 
should  be  continued  and  areas  that  require  improvement  through  additional  training, 
purchase  of  equipment,  and/or  policy  changes.  Although  standardize  templates  for 
conducting  critiques  and  writing  AARs  of  real-world  incidents  do  not  exist,  DHS  does 
have  templates  for  exercises  and  these  elements  are  incorporated  into  the  National 
Preparedness  System.  However,  gaps  in  the  literature  exist  in  regards  to  the  potential  for 
using  fonnal  critiques  and  AARs  from  real-world  incidents  as  tools  to  help  sustain 
specialized  training  and  education  designed  to  prepare  firefighters  for  responding  to 
incidents  caused  by  asymmetric  threats  to  homeland  security. 
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III.  METHODOLOGY 


Policy  analysis  is  the  methodology  selected  for  analyzing  whether  or  not  fonnal 
critiques  and  AARs  of  the  responses  to  real-world  incidents  can  be  used  to  support  the 
long-term  sustainment  of  specialized  homeland  security  training  and  education.  The 
subject  matter  being  addressed  in  this  thesis  is  local  implementation  of  federally- 
prescribed  recommendations.  The  purpose  of  this  analysis  is  to  assess  the  costs  and 
benefits  of  incorporating  a  different  learning  mechanism  into  the  existing  training  and 
education  methods  endorsed  by  the  DHS  as  a  strategy  to  promote  the  sustainment  of 
capabilities  required  to  mitigate  emergencies  caused  by  asymmetric  threats  to  homeland 
security.  Policy  analysis  is  an  appropriate  methodology  for  exploring  opportunities  to 
improve  upon  existing  practices. 

This  policy  analysis  is  based  on  the  claim  that  firefighters  use  comparable  skill 
sets  when  responding  to  certain  types  of  emergencies — such  as  hazardous  materials 
incidents  and  structure  collapses — that  may  originate  from  both  conventional  and 
asymmetric  causes.  To  assess  the  similarities  and  differences  between  the  mitigation 
procedures  and  technical  skills  used  when  responding  to  incidents  of  similar  nature  but  of 
different  origins,  an  analysis  comparing  the  documentation  from  real-world  incidents  and 
the  federally  prescribed  homeland  security  target  capabilities  is  done.  This  approach  is 
consistent  with  the  National  Preparedness  System’s  HSEEP  guidance  in  which 
preparedness  levels  are  assessed  against  the  federally  defined  TCL4  . 

A  modified  cost-benefit  analysis  is  performed  to  analyze  the  data  from  the 
selected  cases  and  comparison  sample.  Efficiency,  process  values,  and  robustness  and 
unprovability  are  the  criteria  selected  for  the  analysis.  Each  criterion  is  evaluated 
independently,  followed  by  an  assessment  of  similarities  and  differences  between  the 
cases  and  comparison  group. 


4  The  capabilities  that  comprise  the  TCLs  are  arranged  according  to  the  five  categories:  common 
capabilities,  prevent,  protect,  respond,  and  recover.  The  capabilities  that  apply  to  the  fire  service  fall 
primarily  within  the  common  capabilities  and  respond  categories. 
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Analysis  of  efficiency  is  necessary  to  assess  the  financial  implications  of  the 
current  and  proposed  policies.  The  factors  analyzed  are  human  resource  and  equipment 
expenditures.  The  results  of  the  analysis  are  judged  in  terms  of  the  number  of  additional 
firefighters  who  could  attend  technical  training  for  developing  tactical  skills  required  to 
mitigate  homeland  security  incidents  if  a  more  cost-efficient  strategy  for  sustaining 
homeland  security  training  and  education  is  adopted. 

Process  values  analysis  (Bardach,  2009,  p.  31)  is  used  to  assess  the  opportunities 
available  under  the  current  and  proposed  policies  for  responders  to  leam  and/or  have  a 
direct  influence  on  changes  to  departmental  procedures.  Each  step  of  the  training, 
exercising,  and  processes  for  conducting  formal  critiques  is  analyzed  to  identify  the 
opportunities  for  responder  participation.  The  analysis  on  strengths  and  weaknesses  of 
the  current  and  the  proposed  policy  options  is  organized  in  terms  of  opportunities  for 
individual  learning,  organizational  learning,  and  surprise  avoidance. 

An  analysis  of  strengths  and  weaknesses  is  also  used  to  assess  robustness  and 
unprovability  of  the  current  and  proposed  policies.  The  components  for  this  analysis  are 
budget,  time  requirements,  administrative  complexities,  and  flexibility  for  change  during 
policy  execution. 

A.  SAMPLE  SELECTION 

There  are  two  issues  addressed  in  this  section  on  sample  selection.  The  first  topic 
discussed  is  the  selection  of  the  fire  department  from  which  the  cases  for  analysis  are 
drawn.  The  second  subject  is  case  selection. 

The  Chicago  Fire  Department  (CFD)  is  the  data  collection  site  chosen  for  this 
thesis  project.  Chicago  is  the  third  largest  city  in  the  United  States  (U.S.  Census  Bureau, 
2002).  However,  commonalities  in  the  organizational  structure5,  demographic 


5  The  CFD  is  run  by  a  Fire  Commissioner,  with  day-to-day  operations  being  managed  by  a  First 
Deputy  Fire  Commissioner.  The  First  Deputy  oversees  the  five  Department  bureaus-  Operations,  Support 
Services,  Fire  Prevention,  Administration,  and  Labor  Relations.  The  Department  operates  over  160  fire 
suppression  and  rescue  apparatus,  six  special  operations  teams,  and  75  ambulances  daily.  The  Department 
responds  to  over  650,000  calls  for  service  annually  (CFD,  2008) 
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characteristics  of  the  population  served6,  geographic  considerations7,  and  the  types  of 
incidents  to  which  the  fire  department  responds  are  found  when  comparing  the 
characteristics  of  the  CFD  to  those  of  urban  fire  departments  from  cities  of  smaller  size. 
Similar  to  urban  fire  departments  throughout  the  country,  the  CFD  complies  with  the 
codes  and  standards  established  by  the  National  Fire  Protection  Association;  creating  a 
means  for  comparing  the  capabilities  among  urban  fire  departments  that  serve  cities  with 
different  population  characteristics  and  physical  attributes.  The  CFD  receives  federal 
homeland  security  grant  funding  to  develop  and  sustain  capabilities  for  responding  to 
homeland  security  incidents  and  conforms  to  all  grant  requirements  for  equipment 
purchases,  training,  and  exercises.8  The  commonalities  found  among  the  characteristics 
of  urban  fire  department  suggest  that  findings  from  this  study,  which  is  based  on 
Chicago-specific  data,  can  be  generalized  and  applied  to  other  urban  fire  departments. 

Four  real-world  emergencies — two  hazardous  materials  incidents9  and  two 
confined  space10,  trench  rescue11  incidents — that  occurred  in  2009  are  the  sample  cases 
selected  for  this  policy  analysis.  A  conventional  cause  triggered  each  incident  and  the 


6  Chicago  is  home  to  2.8  million  residents  who  reside  in  198  economically,  ethnically,  linguistically, 
and  racially  diverse  neighborhoods  (OEMC,  2006). 

7  The  225  square  miles  of  land  that  comprise  Chicago  includes  28  miles  of  Lake  Michigan  shoreline, 
over  150  miles  of  inland  waterways,  8  major  highways,  and  46  movable  bridges.  Chicago,  which  is  one  of 
the  busiest  recreational  harbors  in  the  country,  has  more  than  1335  high-rise  buildings,  multiple 
manufacturing  plants,  and  numerous  entertainment  arenas  where  thousands  of  people  routinely  gather. 
There  are  193  DHS  recognized  critical  infrastmctures/key  resources  within  the  City.  The  City  is  a  national 
hub  for  rail  transportation  since  it  is  the  geographic  location  where  rail  line  tracks  from  the  east  meet  those 
of  the  western  rail  lines.  (OEMC,  2006) 

8  Per  the  grant  requirements,  all  equipment  purchases  must  be  included  as  part  of  the  federal 
Authorized  Equipment  List,  all  grant  funded  training  must  be  pre-approved  by  the  DHS,  and  all  exercises 
must  conform  to  the  HSEEP  guidelines. 

9  The  CFD  classifies  hazardous  material  incidents  according  to  three  levels.  A  Level  I  hazmat  is 
initiated  primarily  for  investigative  activities  and/or  to  mitigate  incidents  involving  small  quantities  of  low- 
potential  materials.  A  Level  11  hazmat  is  initiated  for  confirmed  incidents  involving  a  moderate  to  high 
and/or  large  quantity  of  hazardous  materials,  an  incident  requiring  protective  clothing  above  Level  D  PPE 
(e.g.,  structural  fire  fighting  clothes),  and/or  if  the  incident  requires  an  evacuation  to  be  initiated.  A  Level 
Ill  hazmat  is  initiated  for  extensive  incidents  that  require  additional  manpower  and/or  more  specialized 
equipment  and  supplies  in  order  to  mitigate,  or  if  an  expanded  area  requires  evacuation.  (CFD,  2004) 

10  A  confined  space  is  defined  as  a  space  that  is  large  enough  and  so  configured  that  a  person  can  enter 
and  perform  assigned  work,  has  limited  or  restricted  means  for  entry  or  exit  and  is  not  designed  for 
continuous  human  occupancy  (CFD,  2003). 

1 1  A  trench  is  defined  as  a  narrow  excavation  made  below  the  surface  of  the  earth  that  is  deeper  than  it 
is  wide  and  less  than  15  feet  wide  (CFD,  2003). 
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technical  skills  employed  for  mitigation  are  characteristic  of  what  is  commonly  required 
for  each  incident  type.  Emergencies  involving  both  incident  types  can  originate  from 
conventional  and  asymmetric  causes,  urban  fire  departments  routinely  respond  to  these 
kinds  of  incidents,  and  both  are  included  on  the  TCL.  Furthermore,  EEGs  for  the 
capabilities  associated  with  each  incident  type  have  already  been  developed  by  DHS. 

Selecting  two  different  incident  types  for  the  sample  creates  the  ability  to  compare 
and  contrast  findings.  The  processes  for  mitigating  hazardous  materials  incidents  and 
incidents  requiring  specialized  search  and  rescue  share  some  common  attributes,  but 
largely  require  the  use  of  different  technical  skill  sets.  Hazardous  material  incidents 
typically  require  the  use  of  metering  equipment,  modeling  software,  and  specialized 
personal  protective  equipment  (PPE).  Incident  that  require  technical  search  and  rescue 
commonly  require  the  use  of  construction  materials  (e.g.,  wood,  nails,  saws),  carpentry 
skills,  and  at  times,  heavy  machinery.  Both  require  the  use  of  reference  materials  and 
support  from  emergency  medical  service  practitioners.  Having  a  sample  comprised  of 
two  incident  types  should  increase  the  confidence  of  any  recommendations  that  suggest  a 
wider  application  of  findings  from  the  analysis. 

A  DHS-approved  awareness-level  course  is  the  training  standard  selected  for 
comparison  with  the  expanded  use  of  formal  critiques  and  AARs  that  is  being  proposed. 
The  course,  Emergency  Response  to  Terrorism  -Basic  Concepts,  is  a  standardized 
curriculum  that  is  endorsed  by  the  U.S.  Fire  Administration  that  is  used  to  train  fire 
service  professionals  throughout  the  country.  The  CFD  continues  to  use  this  curriculum 
as  the  baseline  terrorism  awareness  training  for  all  personnel.  The  expenses  for  the 
delivery  of  and  participation  in  this  training  curriculum  are  eligible  expenditures  under 
the  federal  homeland  security  grant  programs. 

A  City  of  Chicago  exercise  conducted  in  December  2009  is  the  exercise  standard 
selected  for  comparison  purposes.  This  multi-day  exercise  involved  several  agencies 
from  different  disciplines,  however,  each  of  the  target  capabilities  and  associated 
objectives  were  exercised  in  isolation  as  standalone  drills.  The  CFD  exercised  target 
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capabilities  for  WMD/Hazardous  Materials  Response  and  Decontamination  and  Search 
and  Rescue  (Land-Based).  The  exercise  met  the  HSEEP  requirements  and  was  paid  for 
with  grant  funding  from  the  DHS. 

B.  DATA  COLLECTION 

1.  Case  Data 

Documentation  sources  for  each  case  include  Computer  Aided  Dispatch  (CAD) 
reports,  electronic  NFIRS  reports,  and  AARs.  In  addition,  the  author  reviewed 
photocopies  of  company  journal  entries  or  performed  on-site  journal  review. 

2.  Training  and  Exercise  Data 

Sources  of  information  for  the  training  course  include  training  rosters  and 
reimbursement  reports.  The  records  include  the  number  of  class  sessions  held,  the  rank 
of  each  instructor,  and  the  amount  of  instructor  compensation.  Planning  records, 
participant  rosters,  the  AAR,  and  reimbursement  reports  are  the  sources  of  exercise  data. 

3.  Financial  Data 

CFD  salary  and  cost-recovery  tables  serve  as  the  basis  for  the  financial  data 
generated.  The  average  salary  for  all  department  personnel  within  a  given  rank  is  used  to 
determine  salary  costs.  The  calculations  for  personnel  ranks  covered  by  collective 
bargaining  units  reflect  contractual  obligations,  resulting  in  the  use  of  overtime  rates  for 
all  activities  that  require  re-hire  or  backfill  of  union  members.  Exempt  rank  personnel 
are  not  entitled  to  an  increased  hourly  wage  for  hours  worked  in  excess  of  nonnal 
obligations;  therefore,  these  costs  are  calculated  at  the  straight  time  rate.  Calculations  for 
on-duty  activities  use  the  cost-recovery  rates. 

An  hourly  apparatus  and  equipment  charge  is  routinely  included  as  part  of  cost 
recovery.  Although  apparatus  and  equipment  are  used  during  homeland  security 
exercises,  grant  funding  is  not  routinely  used  to  pay  for  these  expenses.  Therefore, 
apparatus  and  equipment  costs  are  excluded  from  exercise  expense  calculations. 
Equipment  and  supply  costs  for  purchases  specific  to  an  event  are  included. 
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C.  DATA  COMPILATION  PROCESS 

1.  Exercise  Evaluation  Guides 

The  HSEEP  EEG  for  each  target  capability  is  completed  using  the  documentation 
from  each  case.  Each  piece  of  information  contained  in  the  documentation  is  treated  as 
an  evaluator  observation.  If  observations  are  recorded  in  more  than  one  place,  they  are 
treated  as  a  single  observation.  For  observations  that  reveal  inconsistent  information, 
such  as  differences  in  the  amount  of  time  a  company  spent  on-scene  as  documented  in  the 
CAD  report  and  company  journal,  the  least  subjective  source  is  used. 

To  complete  each  EEG,  the  author  makes  an  assessment  on  whether  or  not  each 
task  is  fully  completed,  partially  completed,  not  completed,  or  if  the  task  did  not  apply. 
Notes  explaining  how  the  information  contained  in  the  case  documentation  support  the 
assigned  measure  are  inserted  under  the  task  description  for  all  tasks  ranked  as  fully  or 
partially  completed.  The  Target  Capabilities  List:  A  Companion  to  the  National 
Preparedness  Guidelines  (DHS,  2007b)  is  referenced  as  necessary  to  clarify  capability, 
activity,  task,  and  measurement  definitions. 

The  author  completes  the  EEG  Analysis  Sheets  by  identifying  the  strengths  and 
weaknesses  of  the  mitigation  activities  contained  in  the  documentation.  The  strengths 
and  weaknesses  are  readily  apparent  for  cases  that  have  a  written  AAR.  The  analysis  for 
cases  without  a  written  AAR  is  based  solely  on  the  interpretation  of  available 
documentation  and  the  critical  tasks  associated  with  each  target  capability. 

2.  Data  Tables 

A  financial  data  table  is  constructed  for  each  case.  Using  the  cost-recovery  tables 
and  CAD  reports,  the  length  of  time  on-scene  for  each  apparatus,  the  number  of 
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personnel  hours  per  apparatus,12’13  and  the  total  cost  per  apparatus  is  calculated.  The 
numbers  in  each  category  for  all  responding  companies  are  added  to  produce  incident 
totals. 

The  subject  matter  of  this  thesis  focuses  on  formal  critiques  and  the  development 
of  AARs  as  a  strategy  to  sustain  specialized  homeland  security  training  and  education. 
Thus,  the  cost-recovery  data  table  is  used  to  establish  the  cost  of  a  formal  critique  for 
each  case.  However,  it  is  important  to  note  that  the  majority  of  participants  in  a  formal 
critique  typically  participate  while  on  duty,  which  means  that  additional  financial  costs 
are  not  incurred14.  Furthennore,  critique  activities  do  not  include  the  perfonnance  of 
mitigation  activities;  precluding  the  eligibility  of  these  activities  for  cost  recovery. 
Although  critique  costs15  are  not  recoverable,  the  efficiency  analysis  includes  this 
information  as  a  cost  comparison  when  discussing  training  sustainability.  Supervisory 
personnel  who  were  not  part  of  the  incident  commonly  participate  in  fonnal  critiques,  but 
are  excluded  from  cost  calculations.  Computation  of  costs  is  based  on  each  critique 
lasting  three  hours16. 

Although  strong  similarities  may  exist,  no  two  incidents  or  exercises  are  exactly 
the  same.  Therefore,  personnel  and  resource  requirements  will  vary  between  incidents 
and  exercises.  As  part  of  the  efficiency  analysis,  an  estimate  of  time,  personnel,  and 


12  One  engineer,  three  firefighters,  and  one  officer  staff  an  engine  company.  Four  firefighters  and  one 
officer  staff  a  truck  company.  Five  firefighters  and  one  officer  staff  a  squad  company.  Two  firefighters 
staff  a  command  van.  One  paramedic  and  one  paramedic  officer  staff  an  ambulance.  Battalion  and 
Paramedic  Field  Chiefs  drive  themselves,  while  Deputy  District  Chiefs  and  Assistant  Deputy  Chief 
Paramedics  have  drivers.  Non-platoon  exempt  rank  personnel  drive  themselves.  Dedicated  staff  are  not 
assigned  to  support  apparatus  but  are  instead  assigned  to  a  team  company.  A  member  of  a  team  company, 
which  is  a  fully  staffed  engine  or  truck  company,  drives  the  support  apparatus  to  an  incident  as  required. 
CFD  is  permitted  up  to  30  manning  variances  per  day  per  the  union  contract.  These  variances  are  not  taken 
into  account  for  purposes  of  this  study.  (Chicago  Firefighters  Union,  Local  2,  International  Association  of 
Firefighters  &  City  of  Chicago,  2003). 

13  Personnel  assigned  to  support  apparatus  are  accounted  for  by  company  of  primary  assignment. 
Equipment  costs  only  are  generated  for  support  apparatus. 

14  It  is  the  CFD’s  practice  that  companies  involved  in  a  formal  critique  remain  in-service  on-the-air. 
However,  to  help  facilitate  the  conduct  of  the  critique,  the  Fire  Alarm  Office  will  prioritize  companies  that 
are  not  participating  in  the  critique  for  dispatch,  within  defined  parameters. 

15  For  purposes  of  calculating  formal  critique  costs,  it  is  assumed  that  all  participants  will  be  present 
for  the  duration  of  the  critique. 

16  CFD  critiques  typically  last  two  hours.  An  additional  hour  is  added  to  reflect  time  spent  traveling  to 
and  from  the  critique  location. 
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associated  expenses  for  an  exercise  of  similar  scope  and  nature  to  the  incident  in  each 
case  is  developed.  The  estimate  comprises  the  same  company  configuration  and 
personnel  numbers  dispatched  for  the  case’s  response.  However,  the  structure  of  an 
exercise  requires  all  participants  to  be  present  for  the  duration  of  the  event.  During  an 
exercise  that  replicates  an  actual  event,  companies  that  were  on-scene  for  a  limited  time 
during  the  real-world  incident  spend  an  increased  amount  of  time  on-scene  in  the  staging 
area  during  the  exercise.  Consequently,  the  time  spent  participating  in  an  exercise  is 
often  not  directly  proportional  to  the  real  world  role  for  some  companies.  Each  exercise 
calculation  is  based  on  the  actual  work  time  of  the  incident  plus  four  hours  for  the 
required  pre-and  post-exercise  activities  (e.g.,  check-in,  safety  brief,  hotwash,  site  clean¬ 
up).  Due  to  the  impracticality  of  using  such  a  large  number  of  in-service  companies  for 
an  exercise  and  the  fact  that  reserve  apparatus  and  re-hired  personnel  are  commonly  used 
for  large-scale  exercises,  the  salary  tables  instead  of  the  cost-recovery  tables  are  used  to 
calculate  personnel  costs.17  Expenses  for  the  development  of  the  exercise  AAR  are  not 
included  since  the  AAR  is  typically  written  by  a  salaried  employee.  However,  it  should 
be  noted  that  writing  an  AAR  typically  takes  12  hours. 

The  data  table  constructed  for  the  homeland  security  class  includes  the  number  of 
sessions  conducted,  the  personnel  hours  required  to  deliver  the  training,  and  instructor 
expenses.  Instructors  taught  on  their  day  off,  which  entitles  them  to  the  prescribed 
overtime  rate  for  their  rank.  The  table  includes  expenses  related  to  participation  in  the 
train-the-trainer  course.  Participants  attended  the  training  while  on-duty  as  part  of 
routine  in-service  training,  so  no  participant  expenses  were  incurred. 

The  costs  for  equipment,  supplies,  and  personnel  comprise  the  homeland  security 
exercise  data  table.  The  total  number  of  personnel  hours  encompasses  the  time  spent 
planning,  executing,  and  evaluating  the  exercise.  Data  is  separated  for  the  hazmat 
incident  and  technical  rescue  incident  components  of  the  exercise. 


1 7  The  officer  position  on  a  truck,  engine,  or  squad  may  be  staffed  by  either  a  lieutenant  or  a  captain. 
The  officer  position  on  an  ambulance  may  be  staffed  by  either  a  paramedic-in-charge  or  an  ambulance 
commander.  For  consistency  purposes,  all  rates  are  based  on  the  suppression  officer  being  a  lieutenant  and 
the  ambulance  officer  being  a  paramedic-in-charge.  Both  titles  are  the  junior  ranking  officers. 
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IV.  CHICAGO  FIRE  DEPARTMENT  INFORMATION  SOURCES 


The  two  sections  in  this  chapter  provide  context  for  the  reader  on  the  data  sources 
and  sample  cases  used  for  this  thesis  project.  The  first  section  details  CFD  processes  to 
capture  incident  response  infonnation.  The  second  section  provides  a  brief  description  of 
each  case  selected  for  the  sample. 

A.  DATA  CAPTURE  PROCESSES 

The  purpose  of  this  section  is  to  validate  the  reliability  of  the  sources  used  for 
data  collection.  With  the  exception  of  the  AARs,  the  data  capture  for  each  source  is 
governed  by  either  a  CFD  general  order  or  Office  of  Emergency  Management  and 
Communications  (OEMC)  Fire  Alarm  Office18  standard  operating  procedures.  CFD  uses 
a  standardized  template  for  AARs,  however,  the  conduct  of  formal  critiques  and 
development  of  AARs  is  not  governed  by  a  general  order. 

Each  time  a  9-1-1  call  is  received  and  a  fire  company  dispatched,  the  Fire  Alarm 
Office  creates  an  incident  ticket  in  the  CAD  system.  The  CAD  system  populates  the 
incident  ticket  with  the  time,  date,  location,  type  of  incident,  and  companies  dispatched. 
Information  captured  for  each  dispatched  company  includes  the  unit  number  and  the 
times  of  dispatch,  departure  from  the  fire  house,  on-scene  arrival,  and  release  from  the 
scene.  Company  officers  press  a  button  on  the  portable  data  terminal  (PDT)  that  is 
installed  in  the  apparatus  to  enter  departure  and  arrival  times.  Since  the  PDTs  are 
connected  to  the  CAD  system,  all  information  inputs  on  the  PDT  are  automatically 
transmitted  to  the  CAD  system  and  entered  in  the  incident  ticket.  Dispatch  personnel 
manually  enter  remarks  in  the  ticket  to  document  reports  provided  by  company  officers 
of  on-scene  conditions,  mitigation  activities,  resource  requests,  and  status  of  response 
operations.  Incident  tickets  are  archived  in  the  CAD  system  as  printable  CAD  reports. 


18  The  OEMC’s  Fire  Alarm  Office  receives  9-1-1  calls  and  is  responsible  for  dispatch  of  all  fire 
companies. 
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Electronic  NFIRS  reports  are  pre-populated  with  data  from  the  CAD  incident 
ticket,  but  require  company  and  chief  officers  to  manually  enter  additional  narrative 
information.  Officers  enter  mitigation  priorities  and  strategies,  the  assignment  of  each 
responding  company,  the  role  of  on-scene  support  agencies,  and  additional  resource 
requests  and/or  notifications  into  the  NFIRS  report.  Some  officers  include  the  tactical 
strategies  used  by  each  company  to  carry  out  their  specific  assignment(s).  Information  on 
injured  victims,  both  those  who  treated  on-scene  and  released  or  transported  from  the 
scene  for  further  medical  care,  is  captured  in  the  NFIRS  report. 

For  incidents  involving  hazardous  materials,  the  Hazardous  Incident  Team  officer 
completes  an  electronic  hazmat-specific  NFIRS  report.  This  report  is  appended  to  the 
main  NFIRS  report.  This  report  contains  information  on  the  type  of  hazardous  materials 
present,  circumstances  and  equipment  involved  in  the  release,  weather  conditions,  tools 
used  for  material  identification,  and  equipment,  supplies,  and  manpower  assignments. 
The  officer  also  writes  a  narrative  detailing  isolation,  identification,  notification, 
mitigation,  and  termination  activities.  Chicago-specific  hazmat  tactical  worksheets  and 
photographs  to  document  the  incident  are  often  incorporated  as  part  of  the  hazmat 
NFIRS. 

Per  CFD  protocol,  each  company  officer  documents  all  of  the  company’s 
activities  for  the  shift  in  the  company’s  leather-bound  journal.  For  each  incident  to  which 
the  company  is  dispatched,  the  officer  documents  the  type  of  incident,  the  CFD  Incident 
Commander,  the  company’s  assignment(s),  the  specific  assignment  of  each  company 
member,  tactics  employed  to  complete  the  assignment,  equipment  used,  and  notifications 
made.  Situation  specific,  journal  entries  may  include  the  activities  of  other  on-scene 
agencies,  injuries  to  responders,  equipment  that  needs  to  be  repaired  or  replaced,  and 
other  unique  circumstances  encountered  or  actions  taken.  All  companies  and  chief 
officers  maintain  journals. 

Chicago  Fire  Department  AARs  contain  a  brief  narrative  of  the  incident,  actions 
taken  for  mitigation,  and  an  analysis  of  what  worked  well  and  areas  for  improvements. 
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Some  CFD  AARs  contain  additional  information  such  as  the  impact  certain  training 
classes  have  had  on  operational  capabilities  and/or  CFD’s  interactions  with  external 
response  partners. 

B.  BRIEF  CASE  DESCRIPTIONS 

1.  Hazardous  Materials  Incident 

a.  Case  1 — Level  II  Hazardous  Materials  Incident  With  an  EMS 
Plan  I19 

On  a  July  weekday,  a  9-1-1  call  reporting  an  explosion  at  a  food 
processing  plant  is  received.  The  initial  responding  companies  confirm  the  report  and 
identify  the  source  of  the  explosion  as  a  large  outdoor  chemical  storage  tank.  A  Level  I 
Hazardous  Materials  Incident  (hazmat)  and  EMS  Plan  I  response  are  initiated;  the  hazmat 
is  subsequently  escalated  to  a  Level  II.  After  conducting  metering  operations  to  identify 
and  establish  hot,  warm,  and  cold  zones,  personnel  trained  as  hazardous  material 
technicians  don  appropriate  PPE  and  enter  the  hot  zone.  PH  strips  and  chemical  spilfyter 
sticks  are  used  to  test  run-off  and  stagnant  water  for  chemical  contamination.  The 
chemicals  involved  are  identified  as  glycerine  and  sulfuric  acid.  On-scene  Fire 
Suppression  and  Rescue  companies  assist  the  Hazardous  Incident  Team  with  deploying 
drain  covers  to  prevent  chemical  run-off  from  entering  the  storm  sewer  and  with 
establishing  emergency  and  technical  decon.  Two  plant  employees  require 
decontamination  and  emergency  medical  treatment  before  being  transported  to  a  hospital 
for  further  medical  evaluation  and  treatment.  One  injured  CFD  member  requires 
decontamination  before  receiving  on-scene  emergency  medical  treatment  and  being 
transported  to  a  hospital  for  further  treatment.  Representatives  from  the  Chicago 
Departments  of  Fire,  Police,  Environment,  and  Buildings,  Common  Wealth  Edison, 
People’s  Gas,  and  personnel  who  worked  in  the  plant  staff  the  unified  command  post. 


19  An  EMS  Plan  I  sends  five  additional  ambulances  to  the  scene,  which  are  above  what  has  already 
been  dispatched  for  the  Level  II  hazmat  (CFD,  2007). 
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The  Fire  Department  Incident  Commander  secures  the  incident  and  turns  it  over  to  the 
Department  of  Environment  for  recovery  operations.  Incident  mitigation  took 
approximately  four  hours. 

b.  Case  2 — Level  II  Hazardous  Materials  Incident 

On  a  weekday  morning  in  January,  a  9-1-1  call  reporting  a  broken 
ammonia  pipe  in  a  cold  storage  facility  is  received.  A  Level  I  hazardous  materials 
response  is  dispatched.  The  companies  who  conduct  the  initial  investigation  confirm  the 
report  and  escalate  the  incident  to  a  Level  II  hazmat.  The  17  employees  in  the  facility  are 
evacuated.  EMS  personnel  triage  and  treat  the  evacuees,  all  of  whom  refuse  transport  to 
a  hospital.  CFD  hazardous  materials  technicians  don  Level  A  PPE20  to  enter  the 
building  and  conduct  metering  operations.  The  incident  is  mitigated  by  shutting  off  the 
supply  valve  to  the  broken  ammonia  supply  pipe.  The  building  interior  is  ventilated  by 
the  pre-installed  exhaust  fans.  Fire  department  personnel  conduct  metering  operations 
outside  the  building  to  ensure  the  safety  of  surrounding  residents  during  building 
ventilation.  One  CFD  member  became  ill  during  the  incident  for  an  unrelated  medical 
issue  and  was  transported  by  EMS  to  the  hospital.  Incident  mitigation  took  two  hours 
and  45  minutes. 

2.  Search  and  Rescue  (Land-Based)  Incidents 

c.  Case  3 — Confined  Space — Trench  Rescue 

On  a  weekday  afternoon  in  May,  a  9-1-1  call  requesting  emergency 
assistance  to  free  a  construction  worker  trapped  in  a  below  grade  hole  is  received.  The 
companies  initially  dispatched  to  the  scene  confirm  that  there  one  person  is  trapped  in  a 
5’  x  5’  x  10’  hole  by  an  earth  cave-in.  The  victim  is  buried  up  to  his  knees  in  an 
unsupported  trench  and  reports  that  his  legs  are  pinned  underneath  a  pipe.  A  full 
confined  space,  trench  rescue  response  is  initiated.  Plywood,  struts,  air  bags,  and  shoring 
are  used  to  stabilize  the  hole.  Shovels  and  a  vacuum  truck  are  used  to  facilitate  the 

Level  A  PPE  is  a  fully  encapsulating  protective  suit  that  is  vapor  and  gas  seal  proof  under  which  a 
self-contained  breathing  apparatus  is  worn. 
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removal  of  the  debris  trapping  the  victim.  While  the  victim  is  still  trapped,  EMS 
personnel  start  an  IV  line  on  the  victim  and  administer  IV  fluids,  pain  medications,  and 
other  medications  as  appropriate  for  a  victim  with  crush  injuries.  The  victim’s  blood 
sugar  level  is  monitored  throughout  the  incident.  An  A-frame  is  constructed  to  assist 
with  packaging  and  removing  the  victim  from  the  hole.  Upon  removal,  the  victim  is 
treated  on-scene  and  transported  to  a  hospital.  External  departments  brought  resources  to 
the  scene  as  requested  to  assist  with  mitigation  operations.  Incident  mitigation  lasted 
approximately  five  hours  and  30  minutes. 

d.  Case  4 — Confined  Space — Trench  Rescue 

On  a  weekend  day  during  August,  a  9-1-1  call  reporting  two  victims 
trapped  in  a  below-grade  hole  is  received.  The  first  companies  on-scene  confirm  that  two 
victims  are  trapped  in  an  unsupported  trench,  with  the  earth  cave-in  trapping  one  victim 
to  just  above  the  waist.  A  full  confined  space,  trench  rescue  response  is  initiated.  The 
trench  is  stabilized  with  plywood,  struts,  and  shoring.  Rigging  ropes  are  attached  to  the 
1st  and  2nd  floor  decks  that  are  above  the  trench  location.  Responders  use  shovels  and  a 
vacuum  truck  to  remove  the  soil.  One  of  the  victims  requires  the  use  of  a  Harty  harness 
and  hoist  for  removal  from  the  trench.  EMS  monitors  the  victims  throughout  the  incident 
and  upon  removal  from  the  trench,  transports  both  to  a  hospital.  Incident  mitigation  is 
completed  in  approximately  six  hours  and  15  minutes. 
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V.  ANALYSIS  AND  FINDINGS 


This  chapter  is  comprised  of  two  sections.  The  first  section  discusses  the 
similarities  and  differences  between  the  response  to  real-world  incidents  and  the  federally 
defined  capabilities  that  are  needed  to  respond  to  homeland  security  incidents.  The 
second  section  details  the  modified  cost-benefit  analysis  conducted  to  assess  whether  or 
not  using  formal  critiques  and  AARs  of  real-world  incidents  is  a  practical  strategy  for 
supporting  the  long-term  sustainment  of  specialized  homeland  security  training  and 
education.  The  modified  cost-benefit  analysis  is  divided  into  three  subsections: 
efficiency  analysis,  process  values  analysis,  and  analysis  of  robustness  and  improvability. 

A.  ANALYSIS  OF  SIMILARITIES  AND  DIFFERENCES 

The  TCL  provides  guidance  to  state  and  local  government  agencies  on  the 
capabilities  deemed  necessary  to  effectively  respond  to  homeland  security  incidents 
(DHS,  2007b).  The  activities  and  tasks  required  to  achieve  each  capability  are  identified, 
as  are  the  specific  actions  perfonned  to  complete  each  task. 

EEGs  for  each  capability  are  structured  so  that  evaluators  can  assess  the 
performance  of  each  activity  by  evaluating  the  task  individually  and  then  recording  an 
overall  assessment  of  progress  toward  achieving  the  capability.  Achievement  of 
activities  is  measured  by  assessing  the  level  of  task  completion  and  the  timeliness  with 
which  target  benchmarks  are  attained.  The  categories  for  task  completion  are  fully 
complete,  partially  complete,  not  complete,  and  not  applicable.  Benchmarks  are 
measured  by  comparing  the  targeted  time  and  the  amount  of  time  actually  required  for 
task  completion.  A  narrative  is  used  to  record  the  overall  assessment  of  strengths  and 
areas  for  improvement. 

To  assess  the  similarities  and  differences  between  incidents  that  originate  from 
conventional  and  asymmetric  causes,  each  of  the  selected  capabilities  and  cases  are 
analyzed  separately,  followed  by  an  assessment  of  whether  or  not  the  case  demonstrates 
performance  of  the  capability.  This  section  concludes  with  a  summarization  of  the 

similarities  and  differences  of  the  findings  for  the  two  capabilities. 
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The  differences  between  data  collection  methods  for  this  study  and  HSEEP- 
compliant  exercises  may  impact  the  assignment  of  level  of  completion  rankings.  During 
an  exercise,  evaluators  observe  player  actions  and  conversations  and  make  note  of  their 
observations  on  the  EEG.  Exercise  evaluators  may  see  or  hear  things  that  help  to  justify 
assigning  a  greater  level  of  completion  than  what  a  company  officer  routinely  documents 
to  indicate  perfonnance  of  the  same  activity.  For  this  thesis  project,  if  documentation 
does  not  exist,  it  is  assumed  that  the  activity  or  task  was  not  perfonned.  Therefore,  some 
of  the  rankings  assigned  may  be  more  restrictive  than  the  rankings  assigned  if  an  EEG  is 
completed  by  a  real-time  evaluator.  Additionally,  exercise  evaluation  plans  permit  the 
exercise  design  team  to  eliminate  some  of  the  TCL  activities  and  tasks  from  the  exercise 
evaluation.  This  thesis  project  assesses  the  level  of  completion  for  all  activities  and  tasks 
for  each  of  the  selected  target  capabilities. 

1.  Weapons  of  Mass  Destruction/Hazmat  Response  and 

Decontamination  Target  Capability 

Eight  activities,  46  tasks,  and  18  time  sensitive  benchmarks  are  identified  for  the 
Weapons  of  Mass  Destruction/Hazmat  Response  and  Decontamination  target  capability. 

a.  Case  1 — Level  II  Hazardous  Materials  Incident  With  an  EMS 
Plan  I 

Case  1  provides  an  opportunity  to  demonstrate  performance  of  all  eight 
activities  that  comprise  the  WMD/Hazmat  Response  and  Decontamination  target 
capability.  Case  documentation  reveals  that  CFD  personnel  fully  completed  35  of  the  46 
required  tasks,  partially  completed  three  tasks,  did  not  complete  four  tasks,  and  that  four 
tasks  are  not  applicable  to  the  circumstances  of  this  case  (see  Table  1).  CFD  met  the 
target  time  for  14  of  the  17  applicable  benchmarks.  Although  performance  of  the 
activities  for  the  three  benchmarks  not  met  is  documented,  the  lack  of  recorded 
timeframes  precludes  an  accurate  assessment  of  whether  or  not  the  target  was  achieved. 

Two  of  the  three  tasks  ranked  as  partially  completed  received  this 
measurement  because  the  required  tasks  are  not  documented  in  the  specified  manner. 

The  EEG  requires  the  development  of  a  written  Incident  Action  Plan  (IAP).  Although 
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the  case’s  plan  of  action  is  documented  in  the  CAD  report,  NFIRS  report,  and  company 
journal  entries,  this  documentation  does  not  meet  the  written  IAP  requirement.  This 
requirement  does  not  account  for  the  fact  that  fonnal  IAPs  are  typically  not  developed  for 
incidents  that  only  last  for  one  operational  period  since  the  incident  is  usually  mitigated 
in  less  time  than  it  takes  to  develop  the  IAP.  While  the  conduct  of  an  informal  critique 
prior  to  companies  being  released  from  the  scene  is  standard  for  the  CFD,  neither  the 
occurrence  of  a  critique  nor  development  of  an  AAR  are  documented  as  required  in  the 
EEG.  The  incident  did  not  require  perfonnance  of  the  third  task  judged  as  partially 
complete  to  the  level  prescribed  in  the  EEG.  Some  of  the  specified  actions  were 
performed  in  a  scaled  manner,  demonstrating  cognitive  ability  to  correctly  identify  the 
need  and  initiate  performance  execution. 

Lack  of  documentation  is  the  reason  that  three  of  the  four  tasks  identified 
as  not  completed  received  this  ranking.  For  the  fourth  task,  plume  modeling,  there  is 
neither  documentation  for  activity  completion  nor  an  explanation  on  why  plume 
modeling  was  not  required. 

This  case  demonstrates  overall  achievement  of  the  target  capability 
WMD/Hazmat  Response  and  Decontamination.  This  case  provides  an  opportunity  for 
responders  to  execute  different  decontamination  techniques,  proactive  protective 
activities,  and  use  different  metering  and  product  identification  equipment. 
Documentation  of  task  performance  indicates  that  the  response  to  this  real-world 
emergency  provided  CFD  personnel  with  an  opportunity  to  execute  the  critical  activities 
and  tasks  deemed  necessary  to  mitigate  hazardous  material  incidents  that  originate  from 
asymmetric  causes. 


39 


Table  1.  Summary  of  Target  Capability  Task  Performance 


Total  Number  of 
Tasks 

Number  of  Fully 
Completed  Tasks 

Number  of 
Partially 

Completed  Tasks 

Number  of  Tasks 
Not  Completed 

Number  of  Tasks 
Not  Applicable 

Number  of 
Benchmarks 

Number  of 
Benchmarks 
Achieved 

Number  of 
Benchmarks  Not 
Applicable 

WMD/Hazmat 
Response  and 
Decontamination 
Case  1 

46 

35 

3 

4 

3 

18 

14 

1 

WMD/Hazmat 
Response  and 
Decontamination 
Case  2 

46 

25 

6 

10 

5 

18 

14 

4 

Search  and 

Rescue  (Land 
Based)  Case  3 

28 

22 

2 

0 

4 

17 

10 

7 

Search  and 

Rescue  (Land 
Based)  Case  4 

28 

21 

3 

0 

4 

17 

10 

7 

b.  Case  2 — Level  II  Hazardous  Materials  Incident 

Case  2  provides  an  opportunity  to  demonstrate  performance  of  all  eight 
activities  that  comprise  the  WMD/Hazmat  Response  and  Decontamination  target 
capability,  but  the  nature  of  the  case  limits  the  number  of  tasks  and  actions  required  for 
incident  mitigation.  Full  completion  of  25  of  the  46  tasks  and  partial  completion  of  six 
tasks  are  contained  in  the  case  documentation.  Ten  of  the  target  capability  tasks  are  not 
completed  and  five  tasks  did  not  apply.  All  applicable  benchmarks  are  met. 

Again,  the  primary  reason  for  a  task  receiving  a  partial  or  not  complete 
ranking  is  lack  of  documentation.  Although  metering  operations  are  documented  and 
metering  is  typically  conducted  to  establish  the  hot,  warm,  and  cold  zones  for  a  hazardous 
materials  incident,  documentation  regarding  the  establishment  of  zones  is  not  found. 
Standard  procedures  for  entry  into  a  hazardous  environment  that  requires  the  use  of  Level 
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A  personnel  protective  equipment  (PPE)  requires  establishment  of  a  back-up  team  and 
emergency  decon.  While  it  is  probable  that  both  occurred,  lack  of  documentation  causes 
these  tasks  to  receive  partial  and  not  completed  rankings.  Development  of  formal  IAPs, 
plume  modeling,  and  the  conduct  of  informal  and  formal  critiques  are  not  documented. 

This  case  demonstrates  achievement  of  seven  of  the  eight  activities  listed 
for  the  WMD/Hazmat  Response  and  Decontamination  target  capabilities.  Insufficient 
documentation  exists  to  determine  whether  or  not  decontamination  operations,  the  eighth 
activity,  for  first  responders  occurred  in  accordance  with  the  required  tasks  prescribed  in 
the  TCL.  This  case  provides  an  opportunity  for  responders  to  operate  in  Level  A  PPE, 
conduct  metering  operations,  and  triage  and  treat  victims.  Documentation  from  this  case 
reveals  that  the  activities  and  tasks  performed  in  response  to  an  incident  originating  from 
a  conventional  cause  are  consistent  with  the  activities  and  tasks  that  DHS  anticipates  are 
needed  to  mitigate  incidents  with  asymmetric  origins. 

c.  Comparison  of  Cases  1  and  2 

Both  cases  provide  opportunities  for  responders  to  execute  the  activities 
and  tasks  that  comprise  the  target  capability  WMD/Hazmat  Response  and 
Decontamination.  However,  neither  case  alone  provides  an  opportunity  to  exercise  ah  of 
the  tasks  associated  with  each  activity.  This  finding  is  reflective  of  HSEEP  allowances 
for  selecting  specific  activities  and  tasks  for  inclusion  in  the  evaluation  that  are  specific  to 
the  exercise  scenario. 

The  level  of  response  required  to  mitigate  each  case  may  be  less  than  what 
is  required  to  mitigate  an  incident  with  an  asymmetric  origin.  For  example,  the  limited 
number  of  people  requiring  decontamination  in  the  cases  could  potentially  be  a  much 
greater  number  during  a  homeland  security  incident.  However,  the  evaluation  of  the 
cases  suggests  that  real-world  incidents  do  provide  opportunities  to  perform  similar  tasks. 
Routine  execution  increases  abilities  to  successfully  perfonn  a  task  under  different 
circumstances  that  may  be  more  complex  in  nature. 

In  both  cases,  documentation  of  certain  tasks  does  not  consistently  meet 
the  standards  established  in  the  target  capability.  Without  observing  on-scene  activities 
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or  interviewing  personnel  involved  in  the  incident  response,  it  is  impossible  to  know 
whether  or  not  the  tasks  were  performed  but  not  documented  or  if  the  tasks  were  just  not 
performed.  There  is  consistency  between  the  two  cases  in  the  items  not  documented, 
such  as  searching  the  scene  for  an  improvised  explosive  device,  plume  modeling,  and 
conducting  informal  critiques.  This  consistency  may  be  the  result  of  the  tasks  being 
standard  in  the  response  to  every  hazardous  materials  incident  and  therefore  not 
documented  as  a  special  activity  performed.  Conversely,  the  lack  of  documentation  may 
exist  purely  because  the  tasks  are  not  performed.  Possible  reasons  for  non-perfonnance 
include  the  lack  of  requirement  by  the  circumstances  of  the  incident,  the  tasks  not  being 
included  in  departmental  operating  procedures,  and/or  neglect  by  the  incident  commander 
or  company  officer.  An  on-scene  evaluator  would  likely  observe  if  such  activities  are 
performed  but  not  documented  and/or  be  aware  of  the  reason(s)  for  non-performance. 
However,  the  structure  of  this  thesis  project  limits  the  extent  of  observations  to  existing 
documentation  only,  impeding  the  ability  to  identify  the  true  causes. 

Overall,  the  cases  suggest  that  the  activities  and  tasks  for  this  target 
capability  are  routinely  performed  when  responding  to  incidents  of  conventional  origin. 
The  tasks  performed  are  customized  to  the  requirements  for  incident  mitigation.  The 
similarities  between  the  responses  to  conventionally  and  asymmetrically  caused 
hazardous  materials  incidents  create  opportunities  for  using  conventionally  caused 
incidents  to  prepare  for  ones  of  asymmetric  origin. 

2.  Search  and  Rescue  (Land-Based)  Target  Capability 

Seven  activities,  28  tasks,  13  time  sensitive  elements,  and  four  numeric  targets  are 
identified  for  the  Search  and  Rescue  (Land-Based)  target  capability. 

a.  Case  3 — Confined  Space — Trench  Rescue 

Case  3  provides  an  opportunity  to  demonstrate  perfonnance  of  six  of  the 
seven  activities  that  comprise  the  Search  and  Rescue  (Land-Based)  target  capability. 
Case  documentation  reveals  that  CFD  personnel  fully  completed  23  of  the  28  required 


42 


tasks,  partially  completed  two  tasks,  and  that  four  tasks  are  not  applicable.  There  are  no 
missed  opportunities  to  complete  a  task.  Ten  of  the  17  benchmarks  are  met,  seven  do  not 
apply. 

One  activity  in  this  target  capability  is  devoted  completely  to  searching  for 
possible  victims.  This  case’s  victim  is  visualized  from  incident  onset,  obviating  the  need 
to  perfonn  search  activities.  In  spite  of  this,  documentation  identifies  the  performance  of 
some  of  the  tasks  associated  with  this  activity  since  search  and  rescue  activities  are 
systematic,  integrated,  and  commonly  share  the  procedures  used  for  scene  control  and 
communications.  For  example,  the  system  used  for  accountability  of  search  personnel 
and  equipment  is  established  since  it  is  the  same  process  used  for  tracking  resources 
allocated  to  rescue  operations.  The  communication  and  planning  functions  that  are 
established  on-scene  for  this  case  and  a  single  visualized  victim  are  consistent  with 
procedures  used  in  support  of  search  activities  for  trapped  and  missing  victims.  The 
available  documentation  points  toward  four  of  the  six  benchmarks  for  this  activity 
identified  as  not  applicable  as  possibly  receiving  a  fully  or  partially  completed  score 
based  on  the  tasks  performed.  However,  since  these  benchmarks  are  related  to  an  activity 
that  does  not  apply  to  this  case,  they  are  also  categorized  as  not  applicable. 

Two  tasks  are  ranked  as  partially  complete.  The  first  receives  this  ranking 
since  the  EEG  requires  the  submission  of  documentation  during  the  course  of  the 
incident.  For  operations  that  only  last  one  operational  period,  CFD  policy  requires 
documentation  to  be  completed  after  the  incident  is  secured.  Case  3  lasts  for  one 
operational  period.  The  second  task  scored  as  partially  complete  is  sanitation 
precautions.  This  ranking  is  assigned  since  the  case  does  not  require  a  significant  number 
of  actions  to  maintain  sanitation.  Documentation  does  exist  in  regards  to  medical  waste 
removal  and  securing  the  scene  prior  to  departure,  but  these  activities  do  not  fully  address 
the  comprehensive  definition  of  sanitation.  Partial  credit  is  given  since  responders 
documented  consideration  of  this  task  and  completion  of  the  limited  actions  required. 

Benchmarks  for  this  target  capability  include  requesting  search  and  rescue 
assistance  from  regional  and  federal  partners  and  extracting  victims  from  different  types 
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of  construction.  Resources  external  to  the  City  are  not  required  for  case  mitigation.  Only 
one  type  of  construction  is  involved  in  this  incident. 

By  and  large,  this  case  demonstrates  performance  of  the  target  capability 
Search  and  Rescue  (Land-Based)  for  the  activities  and  tasks  that  apply  to  the  case. 
Although  the  search  activity  is  not  required,  this  case  provides  an  opportunity  to  execute 
complex  rescue  operations.  Responders  perform  rescue  operations  using  equipment 
specifically  procured  to  enhance  CFD  preparedness  for  mitigating  homeland  security 
incidents.  Responders  demonstrate  application  of  rescue  techniques  learned  while 
attending  specialized  training  that  is  funded  with  Homeland  Security  Grant  Program 
funding.  Documentation  of  task  performance  suggests  that  this  real-world  emergency 
provides  an  opportunity  to  execute  critical  activities  and  tasks  deemed  necessary  by  DHS 
to  mitigate  homeland  security  incidents  that  originate  from  asymmetric  causes. 

b.  Case  4 — Confined  Space — Trench  Rescue 

The  findings  from  Case  4  are  similar  to  the  findings  from  Case  3. 
Performance  of  six  of  the  seven  activities  is  documented  with  2 1  of  the  28  required  tasks 
being  fully  completed.  Case  documentation  identifies  that  three  tasks  are  partially 
completed  and  that  four  tasks  do  not  apply.  Again,  there  are  no  missed  opportunities  to 
perform  at  least  part  of  an  applicable  task.  Of  the  17  benchmarks,  10  are  achieved  and  17 
do  not  apply. 

Similar  to  Case  3,  Case  4  does  not  require  search  activities  and  only  lasts 
for  one  operational  period.  Extensive  action  to  address  sanitation  and  external  resources 
is  also  not  necessary.  Although  issues  and  accomplishments  are  documented  by 
company  officers,  a  fonnal  critique  and  AAR  are  not  documented,  earning  this  task  a 
score  of  partially  complete.  For  reasons  analogous  to  the  ones  listed  for  Case  3, 
documentation  for  Case  4  suggests  that  this  case  demonstrates  achievement  of  the 
applicable  activities  and  tasks  for  the  target  capability  Search  and  Rescue  (Land-Based). 
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c. 


Comparison  of  Cases  3  and  4 


Both  cases  provide  opportunities  to  execute  the  activities  and  tasks  that 
comprise  the  target  capability  Search  and  Rescue  (Land-Based).  Although  neither  case 
requires  search  activities,  the  dual  use  of  support  mechanisms  for  search  and  rescue 
activities  creates  an  opportunity  to  demonstrate  competence  for  certain  functions  related 
to  search  activities.  The  search  activity  and  tasks  not  applicable  to  these  cases  are 
important  skills  that  must  be  mastered  to  fully  achieve  this  capability.  It  is  likely  that  a 
different  type  of  conventionally  caused  incident,  such  as  a  building  collapse,  will  provide 
opportunities  to  perform  these  tasks.  This  finding  highlights  the  flexibility  of  the  HSEEP 
for  selecting  activities  and  tasks  that  apply  to  an  exercise  scenario,  acknowledging  that 
not  all  activities  and  tasks  apply  to  every  scenario. 

The  number  of  victims  in  each  case  is  small.  While  the  victim  number 
does  not  trigger  a  response  on  the  scale  of  a  mass  casualty  incident,  the  cases  provide 
opportunities  to  utilize  the  same  skills  for  victim  rescue  and  medical  treatment,  but  with  a 
lesser  quantity  of  personnel.  Whereas  scene  control  may  become  more  complex  for 
larger  incidents,  additional  supervisory  personnel  are  also  dispatched  for  larger  incidents 
to  maintain  an  effective  span  of  control.  The  National  Incident  Management  System 
(NIMS)  identifies  the  standard  for  span  of  control  as  one  supervisor  for  every  three  to 
seven  subordinates  (Federal  Emergency  Management  Agency,  2005).  Larger  incidents 
may  require  the  establishment  of  additional  levels  of  supervision,  but  tactical  procedures 
and  communication  protocols  remain  unchanged. 

Appropriate  use  of  technical  rescue  techniques  is  a  commonality  found 
between  the  cases.  Specialized  training  and  education  is  required  to  learn  the  appropriate 
rescue  techniques  but  practice  is  required  for  skill  mastery.  Evaluation  of  case 
documentation  in  tenns  of  the  Search  and  Rescue  (Land-Based)  target  capability 
identifies  that  the  execution  of  the  technical  rescue  techniques  perfonned  to  mitigate 
incidents  that  originate  from  conventional  causes  are  the  same  as  the  those  required  to 
mitigate  incidents  caused  by  asymmetric  threats  to  homeland  security.  While  specialized 
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education  and  training  remains  necessary  to  develop  technical  rescue  skills,  the  findings 
from  this  evaluation  suggests  that  real-world  incidents  can  be  used  to  prepare  for  a 
possible  homeland  security  incident. 

3.  Comparison  of  Capabilities 

The  findings  from  the  cases  for  both  the  WMD/Hazardous  Material  Response  and 
Decontamination  and  Search  and  Rescue  (Land-Based)  capabilities  imply  that  similar 
activities  and  tasks  are  used  to  mitigate  incidents  that  originate  from  both  conventional 
and  asymmetric  causes.  Incidents  of  conventional  origin  may  be  of  smaller  scope  than 
those  of  asymmetric  origin,  but  the  mitigation  techniques  and  procedures  are  largely 
consistent. 

None  of  the  cases  provide  an  opportunity  to  demonstrate  performance  of  all 
activities  and  tasks,  which  reflects  HSEEP  planning  assumptions.  Opportunities  to  fully 
complete  some  tasks  are  limited  by  the  size  of  each  incident.  The  benchmarks  are 
consistently  met,  often  being  achieved  in  less  than  half  the  amount  of  the  targeted 
timeframe.  The  geographic  confines  of  the  City  of  Chicago  and  richness  of  CFD’s 
specialized  resources  are  likely  confounding  factors  for  this  finding.  Fire  departments 
that  have  fewer  specialized  resources  may  not  be  able  to  fully  replicate  these  findings. 
However,  departments  that  comply  with  National  Fire  Protection  Agency  response  time 
targets  should  be  able  to  initiate  response,  basic  on-scene  activities,  and  the  request  for 
mutual  aid  resources  required  to  support  specialized  operational  requirements  within  the 
targeted  benchmarks. 

The  cases  selected  to  evaluate  the  WMD/Hazardous  Materials  Response  and 
Decontamination  capability  display  a  greater  variability  among  tasks  than  the  Search  and 
Rescue  (Land-Based)  capability  cases.  This  variance  is  attributable  to  the  hazardous 
material  incidents  originating  from  different  causes — a  chemical  reaction  causing  an 
explosion  vs.  a  broken  pipe  leaking  ammonia,  while  both  trench  rescue  cases  originated 
from  the  same  cause — earth  caving-in  on  an  unsupported  trench.  This  variance 
highlights  the  importance  of  not  using  a  single  incident  or  single  type  of  case  in  isolation 
to  assess  overall  achievement  of  a  capability.  Cases  3  and  4  demonstrate  that  the 
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activities  and  tasks  used  to  mitigate  similar  incidents  are  consistent.  Cases  1  and  2 
demonstrate  that  there  can  be  great  variability  in  the  tasks  required  to  mitigate  incidents 
with  the  same  classification. 

Documentation,  particularly  documentation  of  formal  critiques  and  AARs,  is 
consistently  a  weakness  found  among  cases.  Case  3  is  the  only  case  for  which 
documentation  of  a  formal  critique  and  AAR  is  found. 

Bazennan  and  Watkins  (2004)  assert  that  a  predictable  surprise  will  occur  if  a 
problem  is  allowed  to  worsen  over  time  and/or  the  tendency  to  maintain  the  status  quo 
impedes  adequate  preparation  for  addressing  known  deficiencies.  Evaluation  of  the  cases 
in  terms  of  the  target  capabilities  shows  that  the  tasks  performed  during  the  response  to 
conventionally  caused  incidents  are  consistent  with  the  tasks  required  to  mitigate 
incidents  with  an  asymmetric  origin.  Fire  departments  know  what  activities  and  tasks 
they  need  to  perform  if  called  upon  to  mitigate  an  incident  of  asymmetric  origin. 
Identifying  areas  for  improvement  during  incidents  of  conventional  origin  and  acting 
upon  this  information  to  strengthen  areas  requiring  improvement  can  help  a  department  to 
avoid  a  predictable  surprise. 

B.  MODIFIED  COST-BENEFIT  ANALYSIS 

Three  analysis  criteria  are  addressed  in  this  section:  efficiency,  process  values, 
and  robustness  and  improvability.  CFD  financial  data  is  used  to  assess  efficiency.  The 
U.S.  Fire  Administration’s  recommended  processes  for  conducting  formal  critiques  and 
writing  AARs,  DHS  training  catalog,  and  the  HSEEP  process  for  planning  exercises  are 
used  to  conduct  the  process  values,  robustness,  and  improvability  analysis. 

1.  Efficiency  Analysis 

Efficiency  is  analyzed  in  terms  of  the  four  cases  and  the  DHS-approved 
awareness  level  training  course  and  HSEEP-compliant  exercise  selected  as  the  training 
and  exercise  examples  used  to  demonstrate  the  current  policy.  The  results  of  the  analysis 
are  judged  in  terms  of  the  number  of  additional  personnel  who  could  attend  technical 
training  if  a  more  cost-efficient  means  of  sustaining  specialized  training  and  education  is 
employed. 
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a.  Case  1 — Level  II  Hazardous  Materials  Incident  With  an  EMS 
Plan  I 

Incident  mitigation  took  approximately  four  hours  and  required  113 
responders,  who  comprised  46  companies  (see  Table  2).  Responders  collectively 
invested  226  hours21  in  this  incident.  The  response  activities  cost  $12,642.3622,  of  which 
100%  is  eligible  for  cost  recovery  from  the  business  owner  since  the  incident  resulted 
from  a  violation  of  hazardous  material  handling  standards.  Although  not  conducted,  a 
formal  critique  of  this  incident  requires  an  investment  of  339  additional  personnel 
hours23,  for  a  total  cost  of  $16,120.68  (see  Table  3).  An  exercise  of  comparable  size, 
scope,  and  length  requires  an  investment  of  904  personnel  hours24  and  costs 


21  Personnel  hours  for  each  case  are  calculated  based  on  the  actual  time  each  fire  company  was 
engaged  in  responding  to  the  incident.  The  number  of  hours  per  company  is  multiplied  by  the  number  of 
personnel  assigned  to  the  company.  The  total  personnel  hours  for  all  companies  is  added  together. 

22  Costs  for  each  case  are  calculated  using  the  CFD’s  cost-recovery  salary  tables.  The  number  of 
hours  per  firefighter  and  paramedic  are  multiplied  by  the  average  salary  for  the  designated  rank. 

Equipment  costs  are  calculated  based  on  the  length  of  time  each  piece  of  equipment  is  dedicated  to  the 
response.  The  costs  for  all  personnel  and  equipment  are  added  together. 

23  Time  investments  in  formal  critiques  are  calculated  based  on  the  critique  lasting  two  hours,  plus  an 
additional  hour  for  transit  to  and  from  the  critique  location.  It  is  assumed  that  all  personnel  who  responded 
to  the  incident  will  participate  in  the  critique;  therefore,  the  number  of  incident  responders  is  multiplied  by 
3 -hours  each. 

24  Personnel  hours  for  exercises  are  calculated  based  on  the  duration  of  the  exercise,  plus  an  additional 
four  hours  for  the  pre-  and  post-exercise  activities  that  are  required  by  HSEEP. 
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$44,858.2425.  Additional  costs  for  an  exercise  include  exercise-specific  supplies,  and 
personnel  expenses  to  plan,  control,  and  evaluate  the  exercise. 

b.  Case  2 — Level  II  Hazardous  Materials  Incident 

Case  2  lasted  two-hours  and  45  minutes  and  required  20  companies  staffed 
by  57  personnel  to  invest  a  combined  total  of  68  personnel  hours  to  secure  the  scene.  The 
financial  costs  for  human  and  equipment  resources  eligible  for  cost  recovery  are 
$3,974.74.  A  formal  critique  of  this  incident  requires  an  investment  of  141  personnel 
hours  at  a  total  cost  of  $7,981.08.  A  comparable  exercise  costs  $16,815.74,  requiring  a 
317  personnel  hour  investment. 


25  Exercise  costs  are  calculated  based  on  the  average  overtime  rate  for  the  rank  of  each  participant 
times  the  number  of  hours  of  participation. 
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Table  2.  Resource  Requirements  for  Exercises  of  the  Same  Scope  and  Duration  as  an  Incident 


Length  of 
Incident 

Number  of 
Responders 

Number  of 
Companies 

Number  of 
Personnel- 
Hours 

Total  Cost  of 
Response 

Number  of 

Exercise 

Companies 

Total 

Personnel- 
Hours  per 
Exercise3 

Total  Cost  of 
Exercise 

Case  1 

4  hours 

113 

46 

226  hours 
and  19 
minutes 

$12,642.36 

42 

904 

$44,858.24 

Case  2 

2  hours  45 
minutes 

57 

20 

68  hours  and 

23  minutes 

$3,974.74 

17 

317.25  hours 

$16,815.74 

Case  3 

6  hours 
and  30 
minutes 

99 

36 

439  hours 
and  27 
minutes 

$25,004.82 

28 

987 

$52,870.44 

Case  4 

6  hours 
and  20 
minutes 

98 

33 

343  hours 
and  31 
minutes 

$20,297.88 

26 

840  hours 
and  20 
minutes 

$44,560.65 

DHS-approved 

course 

17  months 

77  sessions 

2250 

(instructors 

only) 

$131,007.70 

HSEEP-  compliant 

exercise- 

Elazardous 

Materials  Incident 

12  hours 

106 

9  companies 
plus  role 
players 

926 

$34,528.28 

HSEEP-compliant 
exercise — Search 
and  Rescue 

Incident 

12  hours 

35 

16 

546 

$25,570.04 

(personnel); 

$24,000 

(supplies) 

Note.  The  number  of  responders  is  the  same  for  the  incident  response  and  the  exercise. 

Tour  hours  are  added  to  the  time  required  for  incident  mitigation  to  account  for  required  pre-and  post-exercise  activities.  Personnel-hours  include  hours  for 
planning,  evaluation,  scene  staging,  and  venue  clean-up. 
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Table  3.  Resource  Requirements  for  Formal  Critiques 


Length 

of 

Critique 

Number  of 
Responders 

Number  of 
Companies3 

Number  of 
Personnel- 
Hours 

Total  Cost 
of  Critique 

Total  Cost 
of 

Comparable 

Exercise 

Case  1 

3 

113 

42 

339 

$16,120.68 

$44,858.24 

Case  2 

3 

57 

17 

141 

$7,228.86 

$16,815.74 

Case  3 

3 

94 

27 

282 

$15,190.44 

$52,870.44 

Case  4 

3 

82 

26 

246 

$12,271.44 

$44,560.65 

aA  lesser  number  of  companies  is  listed  for  attending  the  critique  than  the  number  of 
companies  that  responded  since  some  support  apparatus  are  staffed  by  team  companies. 


c.  Case  3 — Confined  Space — Trench  Rescue 

Mitigations  activities  for  Case  3  lasted  six  hours  and  30  minutes  and  costs 
$25,004.82.  Thirty-six  companies  with  a  combined  total  of  99  responders  invested  439 
personnel  hours  in  response  activities.  Costs  for  this  case  are  recoverable  since  the 
construction  workers  violated  Occupational  Health  and  Safety  Administration  safety 
standards.  The  formal  critique  conducted  required  282  personnel  hours  and  cost 
$15,190.44.  An  exercise  of  comparable  size,  scope,  and  length  requires  987  personnel 
hours,  costing  $52,870.44.  Additional  expenses  are  incurred  for  exercise  planning, 
evaluation,  and  control,  plus  supplies  required  to  properly  stage  the  scene. 

d.  Case  4 — Confined  Space — Trench  Rescue 

The  confined  space-  trench  rescue  for  Case  4  required  98  personnel  on  33 
companies.  The  incident  lasted  six  hours  and  20  minutes,  but  required  an  investment  of 
242  personnel  hours.  Expenses  for  the  response  total  $20,287.88  and  are  recoverable.  A 
formal  critique  requires  246  additional  personnel  hours  and  $13,176.99  while  an  exercise 
of  similar  nature  involves  840  personnel  hours  and  costs  $44,560. 
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e.  DHS-Approved  Awareness  Level  Training  Course 

Seventy-eight  sessions  of  the  class  titled  Emergency  Response  to 
Terrorism:  Basic  Concepts  were  conducted  over  a  17  month  time  period.  Instructors 
invested  an  average  of  28.8  hours  per  session,  with  the  total  instructor  expenses  for  all 
class  sessions  equaling  $131,007.70.  Additional  expenses  are  not  incurred  since  students 
participated  while  on-duty.  Although  opportunity  costs  are  maximized  by  conducting 
this  training  while  members  are  on-duty,  delivery  of  this  class  took  over  a  year  since  it 
had  to  compete  with  other  departmental  training  mandates. 

f  HSEEP -Compliant  Exercise 

The  Operations  Springboard  exercise  consisted  of  a  series  of  drills  on 
consecutive  days  to  test  isolated  target  capabilities.  The  target  capabilities  exercised 
during  this  event  include  the  WMD/Hazmat  Response  and  Decontamination  and  Search 
and  Rescue  (Land-Based)  capabilities. 

Exercise  of  the  target  capability  WMD/Hazmat  Response  and 
Decontamination  required  926  personnel  hours  and  cost  $34,528.28.  Expenses  can  be 
reduced  by  640  personnel  hours  and  $21,254  if  volunteer  role  players  are  used  instead  of 
candidate  firefighters.  However,  the  two  week  time  period  between  the  decision  by  the 
exercise  host  to  conduct  the  exercise  and  the  exercise  dates  precluded  the  CFD  from 
recruiting  volunteers.  Although  volunteers  themselves  may  be  free,  using  volunteers 
does  require  investments  in  recruitment,  training,  and  on-scene  management  activities. 
The  expense  for  the  56  personnel  hours  dedicated  by  40-hour  employees  to  exercise 
planning  and  execution  is  not  included  in  the  financial  costs  since  these  positions  incur 
neither  overtime  nor  require  backfill. 

Approximately  $50,000  was  expended  to  exercise  the  Search  and  Rescue 
(Land-Based)  capability.  Expenses  include  the  cost  of  materials  for  staging  the  scene  and 
mitigation  activities  (e.g.,  nails,  plywood,  spray  paint),  personnel  expenses  for  staging  the 
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scene26  and  player  participation,  as  well  as  charges  for  exercise  planning  and  evaluation. 
The  exercise  required  an  investment  of  546  personnel  hours.  The  92  hours  invested  by 
40-hour  employees  are  included  in  the  personnel  hour  total,  but  not  included  in  the 
financial  expenses  since  these  positions  do  not  incur  overtime  or  require  backfill. 

g.  Discussion 

The  data  suggests  that  conducting  formal  critiques  of  real-world  incidents 
as  a  training  and  education  tool  is  more  economical  than  conducting  a  staged  exercise  to 
practice  a  similar  skill  set.  The  average  cost  of  a  formal  critique  for  the  four  cases  is  34% 
of  the  estimated  cost  of  a  comparable  exercise,  with  a  range  between  28  and  43%. 
Formal  critiques  for  the  hazardous  materials  incidents  cost  an  average  of  34%  of  the 
expenses  incurred  by  the  HSEEP-compliant  exercise,  while  the  search  and  rescue  formal 
critiques  cost  an  average  of  28%  of  the  amount  of  money  spent  on  the  exercise.  There  is 
consistency  between  the  different  incident  types  with  respect  to  the  ratio  of  savings. 

Opportunity  costs  are  optimized  by  conducting  training  and  education 
while  personnel  are  already  on-duty  since  personnel  are  paid  irrespective  of  how  their  on- 
duty  time  is  allocated.  Although  no  additional  costs  are  incurred  for  on-duty  activities, 
assessing  the  issue  in  terms  of  financial  impact  creates  a  mechanism  for  a  meaningful 
comparison  of  options.  Fire  departments  must  operate  within  a  defined  budget,  which  is 
often  lean.  Management  is  frequently  challenged  to  find  ways  for  accomplishing  more 
with  fewer  resources,  necessitating  that  full  advantage  be  taken  of  all  opportunities  for 
cost-efficient  learning. 

Dedicating  personnel  time  to  participation  in  formal  critiques  may 
temporarily  make  a  company  unavailable  for  dispatch.  However,  if  non-participating 
companies  can  “cover”  for  the  limited  time  period  without  operational  capabilities  being 
negatively  impacted,  the  cost  of  this  learning  opportunity  has  been  minimized.  As 
demanded  by  operational  requirements,  a  company  can  be  dispatched  from  the  critique 

26  The  exercise  scenario  required  search  and  rescue  operations  in  a  collapsed  structure.  To  ensure 
safety,  trained  personnel  must  stage  and  secure  the  venue  prior  to  the  initiation  of  exercise  play.  Staging 
the  exercise  venue,  which  consisted  of  one  collapsed  structure  staged  to  permit  internal  access  by  players 
and  three  collapsed  structures  staged  for  external  play  only,  required  240  personnel  hours,  incurring 
personnel  expenses  of  $10,745.76.  This  cost  excludes  the  48  hours  invested  by  40-hour  employees. 
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location  to  the  incident  scene.  While  such  an  interruption  may  negatively  impact  the 
quality  of  the  fonnal  critique  discussion,  it  is  quite  possible  that  an  interruption  will  not 
occur.  On  a  daily  basis,  CFD  companies  spend  part  of  their  shift  drilling  or  attending 
mandatory  on-duty  training  (e.g.,  CPR,  NIMS  compliance)  without  interruption.  If  a 
company  is  dispatched  to  an  incident  during  the  critique,  learning  still  occurs  during  the 
time  company  members  participated  in  the  critique  discussion.  Including  formal  critiques 
as  a  routine  practice  for  organizational  and  individual  learning  does  not  change  daily 
training  requirements  or  incur  additional  expenses,  but  instead  minimizes  the  costs  for 
providing  an  additional  opportunity  for  meaningful  learning. 

When  firefighters  assigned  to  platoon  shifts  attend  specialized  technical 
training  classes,  another  firefighter  must  be  re -hired  at  an  overtime  rate  to  backfill  the 
position  on  the  firefighter’s  assigned  company.27  The  special  operations  training  classes 
attended  by  CFD  personnel  typically  last  five  days,  necessitating  that  two  platoon  shifts28 
be  covered  for  each  student.  The  average  cost  of  tuition  and  backfill  for  a  firefighter  to 
attend  a  hazardous  materials  technician  A  or  B  class  is  $2,591,  with  the  backfill  costs 
increasing  for  students  of  greater  rank.  On  average,  a  class  for  one  of  the  many  technical 
rescue  skills  costs  $3,675  per  student  of  firefighter  rank. 

The  difference  between  the  estimated  exercise  expenses  and  the  critique 
costs  for  the  hazardous  material  incident  cases  is  $38,325,  or  an  average  of  $19,162.50 
per  incident.  Comparing  the  difference  in  savings  between  the  average  cost  of  a  formal 
critique  for  Cases  1  and  2  and  the  expenses  incurred  by  conducting  the  HSEEP-compliant 
hazardous  materials  incident  results  in  a  $22,854  difference. 

For  the  search  and  rescue  cases,  the  difference  between  the  estimated 
exercise  expenses  and  critique  costs  is  $69,980,  or  an  average  of  $34,990  per  incident. 

27  The  Labor  Contract  between  the  Chicago  Firefighters  Union,  Local  2  International  Association  of 
Firefighters  and  the  City  of  Chicago,  Illinois  (2003)  section  16.4  stipulates  the  minimum  requirements  for 
manning. 

28  The  CFD  uses  a  three-platoon  schedule  rotation  for  all  fire  suppression  and  rescue  personnel.  Each 
platoon  shift  lasts  for  24-hours.  Although  some  firefighters  will  only  require  coverage  of  one  platoon  shift 
while  attending  a  training  that  lasts  for  five  consecutive  days,  the  calculations  in  this  paper  are  based  on 
two  shifts  per  individual.  This  ratio  produces  a  lower  number  in  terms  of  the  number  of  personnel  who  can 
attend  training  for  the  specified  amount  of  funding  available,  but  ensures  that  the  number  of  personnel 
identified  is  the  minimum  number  and  that  findings  are  not  over  estimated. 
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The  numbers  remain  consistent  when  calculating  the  number  of  personnel  who  can  attend 
technical  rescue  training  if  the  HSEEP-compliant  exercise  expenses  are  instead  allocated 
for  training. 

In  summary,  conducting  formal  critiques  of  real-world  incidents  that 
require  the  execution  of  capabilities  listed  on  the  TCL  provides  a  cost-efficient  means  for 
maximizing  learning  opportunities.  Since  response  operations  and  formal  critiques  are 
conducted  while  personnel  are  on-duty  and  already  being  paid,  no  additional  financial 
expense  is  incurred.  Learning  objectives  are  still  met,  but  less  expense  is  incurred. 

2.  Process  Values  Analysis 

Analysis  of  process  values  is  conducted  in  terms  of  opportunities  for  individual 
learning,  organizational  learning,  and  surprise  avoidance.  The  strengths  and  weaknesses 
of  the  current  policy — DHS-approved  awareness  level  training  classes  and  HSEEP- 
compliant  exercises — and  the  proposed  policy  option — expanding  the  scope  of  formal 
critiques  and  AARs  from  real-world  emergencies  to  include  “what  if’  questions  as  a 
method  for  sustaining  specialized  education  and  training — are  evaluated.  The  section 
concludes  with  an  assessment  of  the  overall  findings. 

a.  Individual  Learning 

The  current  policy  focuses  primarily  on  a  pedagogical  learning  model. 
For  both  classroom-based  trainings  and  HSEEP-compliant  exercises,  the  designers 
determine  the  areas  of  content  focus.  Although  management  level  staff  have  input  into 
the  content  focus,  frontline  firefighters  often  do  not.  Furthermore,  the  process  for 
selection  of  content  is  usually  not  transparent.  Both  curricula  and  exercise  scenarios  are 
designed  to  address  specific  learning  objectives.  While  exposing  firefighters  to  certain 
information  to  facilitate  learning  is  a  management  responsibility,  overly  structured 
content  may  limit  learning  opportunities  if  the  content  is  not  perceived  by  participants  as 
useful  and/or  adequate  time  is  not  allocated  for  discussing  participant-raised  issues  and 
interests. 
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To  become  a  successful  self-directed  learner,  Knowles  (1989)  contends 
that  adults  must  be  oriented  to  the  model  of  andragogy  since  childhood  education  focuses 
primarily  on  the  pedagogical  model.  The  formal  critique  process  incorporates  the 
recommended  orientation.  At  the  critique  onset,  the  facilitator  leads  the  group  in 
establishing  a  common  operating  picture  by  recreating  the  incident  timeline  (Garvin, 
2000;  Ockershausen,  2008).  Allowing  each  company  to  recount  their  actions  and  address 
facilitator-posed  questions  on  what  worked  and  did  not  work  well  and  the  reasons  why 
creates  opportunities  for  active  engagement  by  participants.  By  the  time  the  session 
progresses  to  the  “what  if’  questions,  participants  are  established  as  active  instead  of 
passive  participants  in  the  learning  process,  completing  the  transition  from  the 
pedagogical  to  the  adragogical  learning  model. 

In  spite  of  the  fact  that  new  infonnation  may  be  learned  during  classroom- 
based  training  and  HSEEP-compliant  exercises,  the  current  policy  does  not  guarantee 
opportunities  for  immediate  application  of  the  infonnation,  contradicting  one  of  Malcolm 
Knowles’  primary  principles  of  adult  learning.  Knowles  postulates  that  immediate 
application  solidifies  information  retention.  If  the  time  frame  between  information 
acquisition  and  the  need  for  application  is  of  extended  duration,  retention  abilities  may  be 
decreased.  This  consequence  positions  learning  to  be  dictated  by  circumstances  that  may 
be  beyond  one’s  control  instead  of  empowering  the  learning  to  take  charge  of  their 
learning  environment. 

The  traditional  formal  critique  discussion  focuses  on  what  went  well,  areas 
for  improvement,  and  changes  that  should  be  made  to  operations  for  a  future  response  of 
similar  nature  (Garvin,  2000;  Ockershausen,  2008).  Expanding  the  critique  discussion  to 
include  “what  if’  questions  for  similar  incidents  with  an  asymmetric  origin  provides 
opportunities  for  immediate  application  of  the  new  information  learned.  Knowles  (1989) 
asserts  that  immediate  application  of  new  information  helps  to  solidify  the  concepts  and 
behaviors  learned.  By  influencing  the  direction  of  the  discussion  and  actively 
participating,  participants  take  responsibility  for  their  own  learning  while  increasing  their 
bank  of  experience  upon  which  to  draw  when  faced  with  a  challenging  situation. 
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Delivery  of  the  DHS-approved  awareness  level  training  course  to  all 
Department  members  required  78  sessions  conducted  over  a  17-month  period  of  time. 
The  structured  curriculum  requires  uniformity  of  delivery  by  the  instructors.  The  time 
constraints  for  delivery  limit  the  opportunities  for  in-depth  discussion  on  areas  of  interest 
to  class  participants,  making  the  class  more  instructor-focused  as  opposed  to  learner- 
focused  as  recommended  by  Kapp  (Peterson,  2009).  The  effect  of  the  repetitive 
curriculum  delivery  by  the  small  group  of  instructors  is  unknown.  Instructors  possibly 
became  more  knowledgeable  about  the  content  areas  during  the  17-month  program, 
which  can  improve  the  quality  of  the  program  delivery.  Conversely,  it  is  also  possible 
that  the  repetition  may  cause  instructor  enthusiasm  to  wane,  leading  to  less  effective 
delivery  of  the  material  and/or  possible  dilution  of  content. 

Action  learning  theory  asserts  that  adults  can  use  their  daily  jobs  to 
exchange  information  with  peers  as  a  means  of  learning  from  collective  experiences 
(Margerison,  n.d.).  The  proposed  policy  capitalizes  on  the  use  of  formal  critique 
participants’  recent  experiences  with  responding  to  a  conventionally  caused  incident  to 
frame  the  discussion  on  responding  to  a  similar  incident  that  originates  from  an 
asymmetric  cause.  Participants  are  provided  with  opportunities  to  actively  direct  the 
learning  process  by  exchanging  information  from  previous  experiences  to  resolve 
proposed  challenges.  Although  firefighters  routinely  respond  to  similar  types  of 
incidents,  the  unique  circumstances  of  each  response — in  terms  of  conditions  of  the 
incident  venue,  the  unique  make-up  of  personnel  staffing  the  responding  companies,  and 
individual  company  assignments — makes  the  experiences  of  each  firefighter  unique.  It  is 
conceivable  that  two  firefighters  of  equal  training  and  assignment  will  learn  different 
lessons  from  the  response  to  the  same  incident.  Providing  a  structured  but  open  forum  to 
exchange  information  on  application  of  activities  and  tasks  to  mitigate  an  incident 
focuses  the  discussion  on  the  key  areas  of  importance  to  the  learner  while  creating 
vicarious  learning  opportunities  for  other  participants. 

Instructor-led  training  programs  require  direct  participation  by  all 
individuals  who  wish  to  leam  the  content  covered.  Although  training  materials  may  be 
made  available  to  non-attendees,  the  opportunities  for  vicarious  learning  are  limited.  The 
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HSEEP  format  for  exercise  AARs  is  neither  user-friendly  nor  conducive  to  use  as  a 
training  tool.  The  current  policy  does  create  learning  opportunities  for  actual 
participants,  but  opportunities  for  vicarious  learning  are  extremely  limited. 

The  AAR  developed  from  the  formal  critique  discussion  creates  a 
vicarious  learning  opportunity  for  firefighters  who  were  not  part  of  the  incident  response 
and/or  the  critique.  This  document  can  be  made  widely  available  as  content  for  company 
schools,  independent  study,  and  reference  material.  Although  the  circumstances  of  the 
incident  are  unique,  those  using  the  AAR  for  vicarious  learning  can  still  relate  to  the 
content  since  there  is  a  high  likelihood  that  the  reader  has  been  involved  in  a  response  to 
a  similar  type  of  incident.  This  association  helps  to  create  a  vivid  image  when  one 
chooses  to  use  an  AAR  as  a  tool  for  self-directed  learning.  Research  by  Irons  (2006) 
demonstrates  that  incidents  with  vivid  outcomes  tend  to  be  more  memorable,  increasing 
the  effectiveness  of  this  learning  strategy. 

b.  Organizational  Learning 

The  current  policy  focuses  primarily  on  improving  existing  practices, 
which  is  commonly  referred  to  as  lower  order  learning  (Espedal,  2008).  Instructor-led 
trainings  and  HSEEP-compliant  exercises  focus  on  developing  and  strengthening 
specific,  pre-determined  capabilities.  These  training  mechanisms  are  important  tools  for 
promoting  the  development  of  core  competencies  required  to  perform  desired 
capabilities.  However,  these  training  methods  commonly  limit  the  opportunities  for 
learning  to  a  fixed  set  of  objectives  (Hohnqvist,  2009).  For  example,  instructor-led 
trainings  are  based  on  a  preset  curriculum  and  HSEEP-compliant  exercises  commonly 
have  artificial  timelines  and  scenarios  that  are  designed  to  test  specific  aspects  of  existing 
policies.  Additionally,  exercise  critiques  involving  all  of  the  participants  are  commonly 
conducted  immediately  following  the  conclusion  of  the  exercise;  limiting  the  amount  of 
time  participants  can  spend  critically  analyzing  events  and  possible  solutions  for  areas  of 
identified  deficiency.  While  the  existing  policy  provides  opportunities  for  organizations 
to  identify  and  correct  areas  of  weakness,  forms  of  lower  order  learning  do  not  promote 
the  creation  of  new  capabilities  (Smith  and  Elliott,  2007). 


58 


The  proposed  policy  of  expanding  the  scope  of  formal  critiques  and  AARs 
to  include  “what  if’  questions  provides  opportunities  for  using  real-world  experiences  to 
critically  evaluate  existing  practices,  experiment  with  making  critical  decisions,  and 
discuss  potential  policy  changes  that  may  be  necessary  to  improve  upon  existing 
capabilities.  This  learning  methodology  is  consistent  with  the  principles  of  higher  order 
learning  as  described  by  Espedal  (2008),  Hohnqvist  (2009),  and  Carmeli  and  Sheaffer 
(2008).  Learning  organizations  must  critically  analyze  issues  and  develop  innovative 
solutions.  The  proposed  policy  provides  a  means  for  front-line  and  management 
personnel  to  work  together  in  a  constructive  manner  to  study  the  cause  and  effect 
relationship  of  actions.  The  intent  of  this  policy  is  to  promote  learning  from  experiences 
and  to  memorialize  effective  practices,  while  recommending  ways  to  improve  upon  areas 
of  defined  weakness  to  expand  existing  capabilities. 

The  training  strategies  embraced  by  the  current  policy  focus  singularly  on 
learning  for  crisis  (Smith  &  Elliott,  2007).  Learning  for  crisis  is  a  fundamental  principle 
of  fire  service  preparedness  initiatives  for  a  jurisdiction’s  specific  hazards.  However, 
Carmeli  and  Sheaffer  (2008)  have  shown  that  learning  from  crisis,  not  learning  for  crisis, 
is  a  key  factor  in  decreasing  the  probability  of  future  crisis  occurrence.  These  researchers 
recommend  that  strategies  for  both  learning  for  and  from  crisis  be  incorporated  into 
training  strategies  aimed  to  provide  opportunities  for  organizational  learning.  Learning 
for  crisis  provides  little  opportunity  for  frontline  personnel  to  contribute  to  overall 
organizational  learning. 

Conversely,  the  proposed  policy  is  designed  to  help  all  levels  of 
organizational  responsibility  to  learn  from  crisis,  the  learning  strategy  that  organizations 
are  most  resistant  to  using  (Smith  &  Elliott,  2007).  The  process  used  to  conduct  formal 
critiques  addresses  the  most  common  barriers  for  organizational  learning  from  crisis: 
availability  of  information,  scape-goating,  and  lack  of  attention  to  lessons  learned,  while 
providing  opportunities  for  contribution  by  all  stakeholders.  The  formal  critique  process 
embraces  the  constructs  for  developing  an  appreciative  learning  culture,  which  has  been 
shown  to  support  organizational  learning  (Barrett,  1995). 
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An  organization’s  ability  to  learn  is  commonly  affected  by  its  capacity  for 
institutionalizing  the  learning  process  and  balancing  the  different  methods  of  learning 
(Smith  &  Elliott,  2007;  Carmeli  &  Sheaffer,  2008;  Hohnqvist,  2009).  Formal  critiques 
and  AARs  cannot  be  used  as  an  isolated  organizational  learning  strategy,  but  must  instead 
be  used  in  combination  with  lower  order  methods  such  as  instructor-led  training  and 
staged  exercises.  The  minimal  to  no  additional  expenses  incurred  when  conducting 
formal  critiques  and  writing  AARs  removes  the  financial  challenges  commonly 
encountered  when  expanding  training  programs.  The  level  of  commitment  by 
organizational  leadership  to  learn  from  crisis  by  institutionalizing  the  proposed  policy  is  a 
primary  consideration  for  whether  or  not  this  strategy  can  be  successfully  incorporated 
into  existing  homeland  security  training  and  education  programs. 

c.  Surprise  Avoidance 

Surprise  avoidance  is  one  of  the  purposes  for  homeland  security  training 
and  education  initiatives.  Although  the  current  policy  does  help  to  prevent  the 
occurrence  of  surprise  by  increasing  one’s  knowledge  in  a  specified  area,  the  policy’s 
effectiveness  is  limited  because  of  the  minimal  stakeholder  involvement  in  accounting  for 
two  factors  known  to  contribute  to  surprise.  Both  noise  barriers  and  the  overestimation 
of  capabilities  can  create  preparedness  vulnerabilities  when  the  current  policy  is  used  in 
isolation  (Handel,  1977). 

Noise  barriers  may  cause  an  organization  to  misallocate  preparedness 
resources.  Noise  can  be  in  forms  such  as  grant  requirements  that  do  not  match  a 
jurisdiction’s  primary  risks,  competing  organizational  responsibilities,  or  the  inability  to 
effectively  process  the  volumes  of  threat-related  infonnation  available  to  accurately 
assess  risks.  The  current  policy  for  classroom-based  training  and  education  limits 
preparedness  funding  expenditures  to  course  curricula  that  have  been  pre-approved  by 
DHS,  largely  excluding  stakeholder  participation  in  content  selection.  Although  the 
DHS-approved  training  catalog  is  extensive,  and  there  are  mechanisms  in  place  to  request 
the  approval  of  additional  curricula,  the  lengthiness  of  the  approval  process  and  time 
requirements  for  implementation  make  selection  of  training  programs  already  approved 
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by  DHS  a  more  time  efficient  decision.  If  the  topics  that  comprise  a  jurisdiction’s 
primary  risks  are  not  included  in  the  catalog,  budget  constraints  and  competing  priorities 
likely  prevent  a  fire  department  from  pursuing  the  new  training,  decreasing  the  possibility 
that  input  from  frontline  personnel  will  influence  training  decisions.  As  discussed  by 
Bazennan  and  Watkins  (2004),  the  tendency  to  maintain  the  status  quo  often  negatively 
affects  abilities  for  effectively  preparing  to  confront  predictable  surprises.  Noise  barriers 
may  influence  the  ability  of  a  fire  department  to  move  beyond  existing  levels  of 
preparedness. 

The  proposed  policy  option  creates  a  means  for  stakeholder  involvement 
in  the  decision  to  memorialize  existing  policy  and  make  recommendations  for  policy 
improvements.  The  collaborative  efforts  to  critique  the  incident  addresses  the  factors 
identified  in  the  Bayesian  theory  (Itti  &  Baldi,  2005)  as  contributors  to  surprise. 
Collaborative  discussion  on  how  to  apply  the  lessons  learned  from  a  real  incident  to  a 
similar  incident  with  a  different  origin  helps  to  decrease  the  uncertainty  of  expectations 
because  the  cause  and  effect  relationship  of  actions  is  studied.  Confidence  in 
expectations  of  actions  by  all  stakeholders  contributes  to  surprise  avoidance  at  both  the 
individual  and  organizational  levels. 

Successful  execution  during  an  exercise  often  equates  to  perceptions  of 
capability  mastery,  which  may  or  may  not  be  accurate  when  attempting  to  execute  the 
same  skills  under  different  conditions.  The  nature  of  exercises  makes  true  simulation  of 
real-world  conditions  difficult.  Practicing  tactical  procedures,  communications,  and 
decision-making  skills  is  beneficial;  however,  it  is  questionable  as  to  whether  or  not 
exercises  truly  provide  opportunities  for  learning  how  to  manage  and/or  avoid  surprise. 
Furthermore,  the  opportunities  for  firefighters  to  influence  the  policy  process  are  limited. 

In  staged  exercises,  planners  routinely  develop  the  scenario  to  test  existing 
capabilities  and  ensure  that  allocated  resources  for  the  event  match  the  requirements  for 
successfully  meeting  exercise  objectives.  The  structure  of  an  exercise  usually  limits  the 
cause  of  the  problem  to  the  pre-defined  scenario  parameters,  limiting  opportunities  to 
practice  comprehensive  assessment  skills  required  to  employ  surprise  avoidance 

strategies  in  real-world  situations  (Henning,  2009).  Injects  may  be  incorporated  into  an 
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exercise  to  evaluate  abilities  to  modify  operations  when  additional  infonnation  is 
received.  However,  the  scenario  and  injects  are  commonly  developed  to  assess  a  limited 
number  of  activities  and  tasks  and  may  not  reflect  realistic  conditions.  Exercise 
participants  may  be  afforded  learning  opportunities,  but  the  breadth  of  opportunity  may 
be  limited  to  the  prc-defincd  scope  of  learning  selected  by  the  planning  team.  As  Itti  and 
Baldi  postulate,  “surprise  is  related  to  the  expectations  of  the  observer”  (2005,  p.  2).  In 
staged  exercises,  participants  are  commonly  briefed  on  the  scenario  in  advance  of  the 
event  and  have  some  forewarning  of  what  to  expect,  limiting  the  ability  to  practice  skills 
for  dealing  with  surprise. 

The  proposed  policy  creates  opportunities  for  firefighters  to  actively 
engage  in  helping  the  organization  to  avoid  surprise  and  learn  how  to  effectively  manage 
situations  in  which  surprise  is  encountered  through  the  creation  of  an  appreciative 
learning  culture  (Barrett,  1995).  By  expanding  the  scope  of  formal  critiques  and  AARs  to 
include  “what  if’  questions,  frontline  personnel  and  administration  are  positioned  to  learn 
collaboratively  from  past  experiences  in  order  to  prepare  for  future  situations  in  which 
similar  challenges  may  occur  under  a  different  set  of  circumstances.  Unlike  a  staged 
exercise  in  which  the  scenario  is  developed  by  a  small  group,  the  scope  of  the  critique 
discussion  is  dictated  by  the  participants  as  they  work  together  to  recreate  the  incident 
timeline  and  identify  strengths  and  weaknesses  of  the  decisions  made  and  tactical 
procedures  employed.  As  recommended  by  Henning  (2009)  and  Irons  (2006),  the 
critique  process  creates  opportunities  to  identify  the  multiple  factors  that  may  contribute 
to  surprise  and  the  tactics  that  can  be  used  to  avoid  surprise  while  optimizing 
participation  by  the  different  levels  organizational  hierarchy  stakeholders. 

The  proposed  policy  option  also  capitalizes  on  the  principles 
recommended  by  Irons  (2006)  to  avoid  surprise.  Stakeholder  involvement  in  identifying 
and  problem-solving  areas  for  improvement  promotes  a  common  understanding  of  the 
overall  mission  while  cultivating  leadership  skills  in  those  of  lower  organizational  rank. 
Engagement  by  frontline  personnel  in  discussions  about  training  needs  helps  management 
to  allocate  resources  to  training  initiatives  that  most  advantageously  create  meaningful 
learning  opportunities  for  the  end  user. 


62 


d.  Summary 


The  analysis  of  process  values  suggests  that  both  policy  options 
successfully  create  learning  opportunities.  The  current  policy  primarily  provides  learning 
opportunities  that  focus  on  content  that  is  pre-detennined  by  a  select  group  of  individuals. 
Conversely,  the  proposed  policy  option  creates  a  mechanism  for  the  actual  learners  to 
influence  both  the  content  to  be  learned  and  organizational  policy.  Therefore,  the 
proposed  policy  is  found  to  be  a  more  advantageous  option  when  process  values  are  a 
priority  consideration  in  the  selection  of  strategies  to  create  an  optimal  learning 
environment. 


3.  Robustness  and  Improvability  Analysis 

Robustness  and  improvability  of  a  policy  are  directly  related  to  field 
implementation  (Bardach,  2009).  The  analysis  of  this  criterion  is  done  as  a  comparison 
between  the  current  and  proposed  policy  options  for  each  of  the  selected  constructs  - 
budget,  time  requirements,  administrative  complexities,  and  flexibility  for  change  during 
policy  execution. 


a.  Budget 

The  current  policy  requires  a  dedicated  budget  for  execution  of  each 
training  and  education  event.  If  federal  funding  is  used  to  support  an  initiative,  all 
expenditures  must  comply  with  federal  requirements.  Using  the  data  from  the  DHS- 
approved  awareness  level  course  and  the  two  HSEEP-compliant  exercises  used  as  case 
comparisons,  the  average  cost  per  personnel-hour  invested  in  the  execution  of  activities 
in  support  of  the  current  policy  is  $52.42.  Reducing  this  cost  may  be  possible  if  a  greater 
number  of  on-duty  personnel  are  used  as  exercise  participants  and  instructors  are  detailed 
from  their  daily  assignments  to  teach.  However,  such  strategies  shift  instead  of  reduce 
the  burden.  On-duty,  exercise  participation  causes  a  large  number  of  companies  to  be 
unavailable  for  dispatch  assignment  for  an  extended  period  of  time,  possibly  creating 
operational  vulnerabilities.  Although  detailing  personnel  to  instructor  positions  does  not 
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negatively  impact  the  training  budget,  the  overtime  budget  for  operations  increases  since 
firefighters  must  be  re -hired  to  fill  vacancies  created  by  detailed  personnel  in  order  to 
maintain  manning  requirements. 

Alternatively,  execution  of  the  proposed  policy  does  not  incur  expenses 
that  are  in  addition  to  the  existing  financial  obligations  required  to  support  daily 
operations.  Fonnal  critiques  are  conducted  and  AARs  developed  as  part  of  routine 
operations  for  on-duty  personnel.  Duplication  of  AARs  for  wide  distribution  does  incur 
expense,  but  the  cost  is  negligible.  Furthermore,  use  of  technology  to  electronically 
distribute  and  archive  documents  reduces  duplication  costs.  Although  there  are  costs 
associated  with  establishing  and  maintaining  technological  systems,  these  systems  are  not 
exclusively  required  for  execution  of  the  proposed  policy  and  should  therefore  not  be 
attributed  as  such. 

b.  Time  Requirements  for  Implementation 

The  HSEEP  doctrine  subscribes  to  a  standardized  template  for  planning 
that  includes  a  series  of  planning  and  coordination  meetings  (DHS,  2007a).  A  minimum 
of  three  meetings  to  plan  an  exercise  and  one  post-exercise  debriefing  meeting  are 
required.  HSEEP  recommends  that  these  meetings  occur  over  a  period  of  many  months. 
Additionally,  an  investment  of  time  is  required  between  meetings  to  coordinate  individual 
agency  operations,  secure  the  venue,  and  acquire  resources  for  exercise  support  (e.g., 
port-a-potties,  props,  consumables).  Time  is  also  required  to  develop  HSEEP  documents 
(e.g.,  situation  manual,  master  event  scenario  list,  EEGs,  safety  briefing,  role  player 
cards)  and  to  stage  the  venue.  The  size  of  the  exercise  impacts  the  number  of  individuals 
involved  in  planning  and  execution  activities,  which  directly  impacts  the  total  number  of 
hours  invested. 

Execution  of  HSEEP-compliant  exercises  requires  that  participants  - 
players,  role-players,  evaluators,  controllers,  and  observers — dedicate  time  in  excess  of 
the  hours  spent  on  exercise  play.  Participants  invest  time  in  picking  up  equipment  before 
traveling  to  the  venue,  completing  designated  check-in  procedures,  attending  a  pre-event 
safety  brief,  participating  in  a  hotwash  immediately  upon  exercise  conclusion,  and 
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cleaning  up  the  exercise  venue  and  returning  equipment  and  supplies  to  the  appropriate 
locations.  Data  from  the  HSEEP-compliant  exercises  included  in  this  study  suggests  that 
four  hours  per  participant  for  pre  and  post  exercise  activities  is  a  reasonable  estimate  of 
additional  time  requirements.  Furthermore,  some  participants  may  have  limited  roles 
during  the  exercise  but  be  required  to  be  present  for  the  event  duration,  creating 
inefficiencies  in  time  usage. 

In  addition  to  hours  spent  on  direct  delivery  of  a  DHS-approved  training 
curriculum,  investments  of  time  are  required  for  attending  train-the-trainer  sessions  and 
coordinating  location  and  participant  logistics.  An  average  of  two  hours  per  instructor 
per  session  in  excess  of  instruction  time  were  required  to  effectively  execute  the  DHS 
course  included  in  this  thesis  project. 

The  U.S.  Fire  Administration  recommends  that  formal  critiques  occur 
within  the  days  immediately  following  the  occurrence  of  the  incident  to  be  critiqued 
(Ockershausen,  2008),  which  limits  the  time  between  the  decision  to  conduct  a  critique 
and  actual  execution.  While  an  investment  of  time  is  required  to  prepare  for  facilitating 
the  critique  discussion,  this  investment  is  commonly  limited  to  a  few  hours  by  a  few 
individuals  since  re-creating  the  event  timeline  and  scenario  and  conducting  analysis  is 
accomplished  by  the  participants  during  the  critique  discussion.  Producing  the  AAR  does 
take  time,  but  again  this  investment  is  typically  limited  to  a  few  individuals. 

c.  Administrative  Complexities 

Execution  of  both  the  current  and  proposed  policies  involves 
administrative  complexities.  But  the  complexities  of  each  are  different. 

To  maintain  operational  capabilities,  HSEEP-compliant  exercises 
commonly  require  that  personnel  be  rehired  as  exercise  participants.  Procedures  for  re¬ 
hiring  CFD  union  employees  are  governed  by  labor  contract  rules,  which  treat  individuals 
of  identical  rank  the  same,  irrespective  of  specialty  training  level.  Homeland  security 
exercises  frequently  require  participation  by  firefighters  with  some  level  of  advanced 
knowledge  and  training.  This  requirement  necessitates  that  management  confer  with 
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Union  officials  to  negotiate  special  re-hire  arrangements  in  order  to  avoid  the  financial 
expense  of  personnel  grievances  for  non-compliance  with  union  contract  rules. 
Negotiation  requires  a  time  investment  and  possibly  concessions  on  a  different  issue  in 
order  to  optimize  the  benefit  gained  from  exercise  participation.  Grant  funds  cannot  be 
used  to  pay  grievances  related  to  an  exercise  that  are  found  in  favor  of  the  plaintiff  even 
if  a  grant  was  used  as  the  funding  source  for  the  exercise;  requiring  that  corporate  funds 
be  allocated  for  this  additional  expense. 

Personnel  assignment  to  attend  formal  critiques  also  requires  scheduling 
coordination  to  ensure  that  operational  capabilities  are  maintained.  However,  the 
procedures  to  accomplish  this  are  no  different  than  those  executed  for  assigning 
companies  to  an  incident  and  changing  the  location  of  non-assigned  companies  to  ensure 
coverage  throughout  the  jurisdiction.  Dispatch  personnel  frequently  execute  these 
procedures,  which  do  not  require  advanced  planning  and  negotiations  since  standing 
protocols  already  exist  to  facilitate  dispatcher  decision-making. 

HSEEP  recommends  that  the  exercise  planning  process  be  completed  over 
a  number  of  months  (DHS,  2007a),  with  6-12  months  being  a  realistic  timeframe. 
Although  this  recommendation  provides  the  time  necessary  to  fully  develop  the 
objectives,  scenario,  and  coordinate  logistics  with  all  stakeholder  agencies,  the  extended 
timeframe  also  increases  the  probability  that  changes  in  organizational  priorities  and 
exercise  design  team  members  will  occur.  Changing  priorities  may  lessen  the  resources 
that  are  available  for  dedication  to  an  exercise  or  make  the  need  to  conduct  the  exercise 
obsolete.  Personnel  changes  increase  time  investments  since  new  members  must  be 
briefed  on  historical  planning  and  decisions,  as  well  as  possibly  re-negotiating  the 
exercise  objectives  and  scenario  to  incorporate  the  new  member’s  input. 

Duplication  and  distribution  of  AARs  follows  the  existing  process  for 
distribution  of  internal  paperwork.  Hardcopies  of  AARs  can  reach  all  fire  houses  on  the 
same  day  they  are  duplicated.  Use  of  technology  to  distribute  AARs  electronically 
expedites  the  process. 
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d.  Flexibility 


The  proposed  policy  is  more  flexible  than  the  current  policy  in  terms  of 
implementation  and  meeting  the  learning  needs  of  those  affected.  HSEEP-compliant 
exercises  and  delivery  of  DHS-approved  curricula  are  frequently  dependent  on  the 
availability  of  specific  resources — dates,  venue,  stakeholders,  and  resources — and  these 
events  are  routinely  planned  to  accommodate  a  negotiated  scheduled.  If  an  event  must  be 
cancelled,  the  process  for  coordinating  all  elements  to  reschedule  within  the  desired 
timeframe  can  be  a  very  complex  process  and  often  involves  re-arrangement  of  other 
commitments.  Alternately,  fonnal  critiques  are  scheduled  in  close  proximity  to  the  date 
of  the  incident  being  critiqued,  facilitating  the  ability  of  critique  planners  to  select  a  time 
and  date  that  is  sensitive  to  current  operational  schedules  and  conducive  to  accomplishing 
the  critique  goals  and  objectives.  If  an  emergency  necessitates  that  the  critique  be 
postponed,  it  is  likely  that  the  group  can  reconvene  later  in  the  day  and/or  be  re-scheduled 
for  the  subsequent  workday  since  department  administration  can  make  decisions  on 
resource  allocation.  If  companies  are  needed  for  dispatch  during  the  critique,  the  rest  of 
the  participants  can  still  continue  during  their  absence.  The  focus  of  the  discussion  may 
change,  but  learning  will  still  occur. 

Delivery  of  DHS-approved  curricula  requires  instructors  to  cover  the  pre¬ 
approved  content  and  provides  little  flexibility  for  the  instructors  to  change  the  course 
content.  Often  times,  only  minimal  time  for  questions  or  for  deeper  exploration  of  areas 
of  special  interest  to  participants  is  incorporated.  Conversely,  the  format  of  structured 
but  open  dialog  recommended  for  formal  critiques  provides  opportunities  for  greater 
discussion  on  areas  of  particular  concern  to  participants  (Ockershausen,  2008),  embracing 
the  concept  of  adult  learning  theory  supported  by  Kapp  (Peterson,  2009),  Revans 
(Margerison,  n.d.),  Knowles  (1989),  and  Margerison  (2005).  Additionally,  if  the 
facilitator  determines  that  the  critique  is  not  progressing  in  a  constructive  manner  or  that 
learning  is  not  being  optimized,  the  critique  format  allows  for  the  facilitator  to  change  the 
direction  of  the  conversation  to  topic  areas  that  may  better  promote  participant  learning. 
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e.  Summary 


The  findings  from  this  analysis  suggest  that  the  proposed  policy  is  more 
robust  and  provides  greater  opportunity  for  improving  upon  the  process  mid-course  than 
the  current  policy.  Fonnal  critiques  and  AARs  create  minimal  financial  burden,  require  a 
limited  amount  of  time  for  pre-  and  post-event  activities,  and  are  flexible  to  accommodate 
both  learner  needs  and  organizational  responsibilities  in  a  simple  and  straightforward 
manner.  HSEEP-compliant  exercises  and  DHS-approved  training  courses  do  provide 
learning  opportunities,  but  also  require  execution  in  a  manner  that  is  neither  as  robust  nor 
flexible  in  implementation  as  formal  critiques  and  AARs. 

C.  CHAPTER  SUMMARY  AND  CONCLUSIONS 

In  Chapter  I,  an  analysis  of  the  similarities  and  differences  between  the  response 
to  real-world  incidents  and  the  federally  defined  capabilities  that  are  needed  to  respond  to 
homeland  security  incidents  is  conducted.  The  four  cases  are  examined  to  assess  whether 
or  not  the  targeted  activities  and  tasks  are  performed  and  the  benchmarks  met  in  a  manner 
that  is  consistent  with  prescribed  standards.  The  findings  from  this  analysis  reveal  that 
the  real-world  incidents  do  provide  opportunities  for  executing  the  same  tactics  and 
procedures  that  DHS  predicts  are  needed  to  mitigate  a  homeland  security  incident, 
although  on  a  smaller  scale. 

Consistent  with  the  HSEEP  doctrine,  none  of  the  incidents  in  isolation  provided 
an  opportunity  to  perform  all  activities  and  tasks  associated  with  a  specific  capability. 
The  variability  provided  by  the  different  case  types  to  execute  different  activities  and 
tasks  suggests  that  the  proposed  policy  should  integrate  the  use  of  multiple  incident  types 
and  causes  of  origin  within  a  single  incident  type  category  in  order  to  optimize  learning. 

Documentation  of  a  fonnal  critique  and  AAR  exists  for  only  one  of  the  four  cases, 
suggesting  that  opportunities  for  learning  from  crisis  are  not  consistently  being  used 
advantageously.  The  process  of  learning  from  crisis  can  help  both  individuals  and 
organizations  identify  areas  of  deficiency  and  initiate  corrective  actions,  decreasing  the 
possibility  of  a  predictable  surprise  occurring. 
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In  summary,  the  analysis  in  Chapter  I  found  that  there  are  many  similarities 
between  the  response  activities  for  similar  type  incidents  that  originate  from  both 
conventional  and  asymmetric  causes.  The  exploitation  of  learning  from  conventionally 
caused  incidents  can  create  opportunities  for  preparing  to  respond  to  incidents  with  an 
asymmetric  origin  that  may  be  of  greater  size. 

A  modified  cost-benefit  analysis  is  conducted  in  Chapter  II.  The  three  criteria 
used  in  the  analysis  are  efficiency,  process  values,  and  robustness  and  improvability. 
Efficiency  is  analyzed  using  CFD  financial  data  for  the  four  cases  and  examples  of 
current  policy — DHS-approved  awareness  level  training  and  HSEEP-compliant 
exercises.  Process  values  and  robustness  and  improvability  are  analyzed  in  terms  of  the 
federal  recommendations  for  conducting  formal  critiques,  DHS-approved  training 
classes,  and  HSEEP-compliant  exercises. 

The  findings  from  the  efficiency  analysis  put  forth  that  conducting  formal 
critiques  of  real  world  incidents  is  a  more  economical  method  of  training  and  education 
than  conducting  staged  exercises  that  are  designed  to  practice  similar  capabilities.  Since 
both  incident  response  and  formal  critique  activities  are  conducted  while  firefighters  are 
on-duty  and  already  being  paid,  opportunities  for  education  and  training  are  increased 
without  incurring  additional  expense.  Utilizing  formal  critiques  and  AARs  from  real- 
world  incidents  is  an  efficient  means  for  sustaining  specialized  training  and  education  to 
prepare  for  responding  to  homeland  security  incidents. 

The  process  values  analysis  supports  the  notion  that  the  proposed  policy  provides 
greater  opportunity  than  the  current  policy  for  stakeholder  influence  on  individual  and 
organization  learning,  as  well  as,  surprise  avoidance.  The  proposed  policy  capitalizes  on 
the  principles  of  andragogy  to  encourage  learners  to  take  responsibility  for  their  learning 
through  active  engagement.  The  format  used  to  conduct  formal  critiques  promotes 
investigating  topic  areas  that  are  of  particular  interest  to  the  actual  learners,  as  opposed  to 
the  content  focus  being  selected  by  a  small  number  of  management  personnel  like  is  seen 
with  the  current  policy.  Using  AARs  as  training  tools  creates  opportunities  for  vicarious 
learning  both  immediately  and  in  the  future.  Firefighters  can  choose  to  independently  use 
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these  documents  at  any  point  to  further  their  personal  bank  of  knowledge.  Formal 
critiques  and  AARs  provide  multiple  opportunities  for  stakeholders  to  engage  in  and 
influence  the  learning  process. 

Analysis  of  robustness  and  improvability  reveals  that  the  proposed  policy  is  a 
more  viable  option  in  terms  of  budget,  time  requirements  for  implementation, 
administrative  complexities,  and  implementation  flexibility.  Consistent  with  the  findings 
from  the  efficiency  analysis,  formal  critiques  and  AARs  are  a  more  economical  option 
than  traditional  classroom-based  training  and  HSEEP-compliant  exercises.  The  time 
required  for  implementation  is  also  less,  with  the  shorter  duration  between  the  decision  to 
hold  an  event  and  event  execution  improving  execution  abilities.  Participant  scheduling 
to  attend  each  of  the  training  option  presents  some  challenges,  but  the  current  policy 
requires  that  special  procedures  be  implemented  while  the  proposed  policy  utilizes 
procedures  used  by  the  organization  on  a  daily  basis.  Finally,  the  proposed  policy  is 
more  advantageous  in  terms  of  flexibility  than  the  existing  policy  since  those  in  charge  of 
execution  have  more  freedom  to  customize  the  delivery  in  order  to  more  fully  address  the 
individual  needs  of  the  group. 

In  conclusion,  the  analysis  contends  that  the  proposed  policy  provides  greater 
benefit  for  the  investment  of  resources  than  the  current  policy  option.  The  proposed 
policy  is  more  efficient  in  terms  of  financial  cost  and  time  investments.  It  also  creates 
greater  opportunities  for  stakeholders  to  influence  individual  and  organizational  learning, 
as  well  as  encourages  firefighters  to  assume  a  greater  responsibility  for  their  own 
learning.  Lastly,  implementation  of  the  proposed  policy  is  more  robust  than  the  current 
one  and  provides  the  ability  for  customization  to  meet  the  specific  learning  needs  of 
stakeholders. 
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VI.  RECOMMENDATIONS 


The  extent  to  which  fonnal  critiques  and  AARs  can  be  used  as  training  tools 
throughout  an  organization  is  only  limited  by  the  user  group’s  level  of  creativity; 
providing  a  tool  that  empowers  firefighters  to  build  upon  their  existing  bank  of 
knowledge  to  enhance  individual,  fire  company,  and  organizational  capabilities  for 
responding  to  incidents  of  asymmetric  origin.  The  incorporation  of  the  use  of  formal 
critiques  and  AARs  from  the  response  to  conventional  incidents  into  existing  homeland 
security  training  and  education  strategies  creates  a  dynamic  opportunity  for  sustaining 
existing  capabilities  while  building  new  ones.  Integration  of  these  smart  practices 
(Bardach,  2009)  into  existing  training  and  education  programs  helps  to  create  a 
decentralized  system  for  sustaining  homeland  security  training  and  education  that  utilizes 
both  instructor-led  and  learner-focused  opportunities  (Brafman  &  Beckstrom,  2006).  A 
decentralized  training  approach  is  likely  better  able  to  withstand  the  ever-present 
challenges  encountered  in  administering  training  and  education  programs  within  the  fire 
service  because  the  diverse  tactics  used  in  a  mixed-methods  approach  are  dependent  on 
different  enabling  factors,  such  as  time,  money,  and  personnel  resources.  The  inability  to 
implement  one  tactic  does  not  preclude  the  execution  of  another. 

The  analysis  of  the  two  target  capabilities  provides  support  for  this  thesis’  claim 
that  firefighters  use  comparable  skill  sets  when  responding  to  certain  types  of 
emergencies  that  may  originate  from  both  conventional  and  asymmetric  causes. 
However,  a  single  conventionally  caused  incident  of  typical  scope  and  nature  alone  is 
likely  of  smaller  scale  than  a  homeland  security  incident  and  does  not  provide  the 
opportunity  to  execute  all  activities  and  tasks  for  the  designated  capability.  This  finding 
supports  the  recommendation  to  incorporate  fonnal  critiques  and  AARs  into  daily 
operations  instead  of  using  them  sporadically. 

Expansion  of  the  scope  of  formal  critiques  and  AARs  to  include  “what  if’ 
questions  about  variables  that  may  occur  during  incidents  of  similar  nature  but  of 
different  origin  creates  a  value  innovation  (Kim  &  Mauborgne,  2005)  for  homeland 

security  training  and  education  (see  Figure  1).  Value  innovation  is  a  business  strategy 
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term  used  by  Kim  and  Mauborgne  (2005,  p.  16)  to  describe  actions  that  favorably  affect 
both  cost  structure  and  the  value  to  the  customer.  Cost  saving  are  made  by  eliminating 
and  reducing  factors  that  the  business  competes  on.  Customer  value  is  increased  by 
raising  and  creating  elements  that  the  industry  has  never  offered. 

Formal  critiques  and  AARs  from  conventional  emergencies  used  as  training  and 
education  tools  to  help  firefighters  prepare  for  responding  to  asymmetric  incidents  creates 
a  new  value  proposition  (Johnson,  2010)  for  sustaining  homeland  security  training  and 
education.  A  value  proposition  is  a  term  used  in  business  models  that  identifies  a  job  that 
needs  to  be  done  and  evaluates  the  possible  options  for  accomplishing  the  job  in  a 
manner  that  is  efficient,  reliable,  and  reproducible.  In  this  case,  the  job  to  be  done  is  the 
provision  of  homeland  security  training  and  education  to  firefighters.  The  possible 
options  for  meeting  the  learning  needs  are  classroom  training,  HSEEP-compliant 
exercises,  and  expanded  formal  critiques  and  AARs  that  include  “what  if’  questions 
about  variables  that  may  occur  during  similar  incidents  that  originate  from  different 
origins.  The  findings  in  this  thesis  project  suggest  that  formal  critiques  and  AARs 
embody  the  three  criteria  for  a  value  proposition  in  a  way  that  is  more  advantageous  to 
fire  departments  for  sustaining  specialized  training  and  education  than  the  more 
traditional  options  for  training  and  education. 

A  strategy  canvas  (Kim  &  Mauborgne,  2005)  is  another  tool  used  in  business 
models  to  pictorially  graph  the  differences  among  possible  options  in  terms  of  each 
option’s  the  level  of  offering  for  the  primary  factor  under  consideration.  The  main 
component  of  a  strategy  canvas  is  the  value  curve,  or  the  graphic  depiction  of  each 
possible  option  in  relation  to  the  other  options.  As  seen  in  Figure  1,  using  expanded 
formal  critiques  and  AARs  from  conventional  emergencies  as  tools  for  preparing 
firefighters  to  respond  to  incidents  with  asymmetric  origin  redefines  the  existing 
homeland  security  training  and  education  value  curve. 
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Figure  1 .  Homeland  Security  Training  and  Education  Strategy  Canvas 

A.  VALUE  CURVE  REDEFINITION 

Value  curves  are  redefined  by  completing  four  actions:  eliminating  factors  that 
the  industry  takes  for  granted;  reducing  factors  well  below  the  industry  standard;  raising 
existing  factors  above  the  industry  standard;  and  creating  factors  that  the  industry  has 
never  offered  (Kim  &  Mauborgne,  2005)  (see  Figure  2).  Expanded  formal  critiques  and 
AARs  accomplish  these  four  actions  by:  eliminating  the  dependence  of  grant  funding  to 
create  learning  opportunities;  reducing  resource  requirements  and  the  probability  of 
predictable  surprise  occurrence;  raising  the  number  of  individual  and  organizational 
learning  opportunities  and  the  number  of  firefighters  impacted  by  each  learning 
opportunity;  and  creating  a  sustainable  homeland  security  training  and  education  strategy 
that  promotes  an  andragogy-based  learning  environment  and  opportunities  for  front-line 
personnel  to  participate  in  policy  development. 
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Eliminate 

•Dependence  on  grant  funding  for 
creating  learning  opportunities 

Raise 

•Number  of  opportunities  for 
individual  learning 
•Number  of  opportunities  for 
organizational  learning 
•Number  of  responders  impacted  by 
each  learning  opportunity 

Reduce 

•Resource  requirements 
•Probability  of  occurrence  of 
predictable  surprises 

Create 

•Sustainable  homeland  security 
training  and  education  strategy 
•Andragogy-based  learning 
environment 

•Opportunities  for  front-line  personnel 
to  participate  in  policy  development 

Figure  2.  Eliminate-Reduce-Raise-Create  Grid 

1.  Eliminate 

Formal  critiques  and  AARs  from  conventional  emergencies  eliminate  the 
dependence  on  dedicated  funding  sources  for  the  creation  of  learning  opportunities. 
Firefighter  salaries  and  equipment  costs  are  expenses  included  in  a  department’s  annual 
budget.  These  expenses  remain  irrespective  of  the  types  of  on-duty  activities  in  which 
firefighters  engage.  A  change  in  the  level  of  Homeland  Security  Grant  Program  funding, 
the  funding  source  that  is  commonly  relied  upon  by  fire  departments  to  support  homeland 
security  activities,  does  not  negatively  impact  abilities  to  conduct  formal  critiques  and 
write  AARs. 

2.  Reduce 

Expanded  fonnal  critiques  and  AARs  reduce  the  human  and  financial  resources 
invested  in  creating  learning  opportunities  that  help  firefighters  prepare  for  responding  to 
incidents  of  asymmetric  origin.  The  expenses  of  instructor  fees  and  participant 
backfilFovertime  are  reduced  since  the  frequency  of  need  for  standalone  classroom-based 
training  is  decreased.  Similarly,  personnel  and  equipment  costs  associated  with  HSEEP- 
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compliant  exercises  are  reduced  since  the  need  to  conduct  exercises  to  practice  activities 
and  tasks  that  are  routinely  executed  in  the  course  of  conventional  fire  service  activities  is 
decreased.  The  new  strategy  does  still  require  an  investment  of  personnel  time  to  prepare 
for  a  critique  and  write  the  AAR,  but  this  investment  is  less  than  the  typical  amount  of 
time  invested  in  planning  and  coordinating  department-wide  training  initiatives  and 
HSEEP-compliant  exercises. 

Formal  critiques  and  AARs  position  frontline  personnel  and  administration  to 
learn  collaboratively  from  past  experiences  in  order  to  prepare  for  future  situations.  The 
expanded  critique  process  creates  opportunities  to  identify  the  multiple  factors  that  may 
contribute  to  surprise  and  discuss  the  procedures  and  tactical  decisions  that  can  reduce 
the  possibility  of  encountering  surprise  during  incident  mitigation  activities.  Bayesian 
theory  postulates  that  surprise  can  only  occur  in  the  presence  of  uncertainty  (Itti  &  Baldi, 
2005).  Routine  engagement  in  individual  and  organizational  learning  reduces  uncertainty 
of  abilities  to  make  decisions  and  execute  activities  and  tasks  during  situations  of 
complex  nature,  ultimately  reducing  the  probability  if  experiencing  a  predictable  surprise. 

3.  Raise 

Expanded  formal  critiques  and  AARs  raise  the  number  of  opportunities  for 
individual  and  organizational  learning.  The  resource  investments  required  for  the 
delivery  of  a  DHS-approved  training  curriculum  and  the  conduct  of  an  HSEEP-compliant 
exercise  limit  the  number  of  events  that  can  be  supported  per  year,  in  turn,  limiting  the 
frequency  and  scope  of  opportunities  to  leam.  Conversely,  formal  critiques  and  AARs 
provide  an  occasion  to  leam  each  time  an  incident  requires  the  use  of  some  of  the 
activities  and  tasks  contained  on  the  TCL.  The  breadth  of  incident  types,  activities,  and 
tasks  covered  during  a  fonnal  critique  is  dictated  by  a  fire  department’s  capabilities  and 
the  types  of  incidents  to  which  fire  companies  are  routinely  dispatched.  This  variety  is 
likely  greater  than  the  focused  content  of  pre-planned  events,  enabling  each  AAR  to 
contain  multiple  learning  points. 

The  number  of  firefighters  impacted  by  each  learning  opportunity  is  also  raised. 
The  ability  to  conduct  formal  critiques  and  write  AARs  is  dictated  by  the  unpredictable 
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occurrence  of  specific  types  of  emergencies.  However,  it  is  this  author’s  experience  that 
completing  a  minimum  of  six  formal  critiques  and  AARs  in  a  12-month  time  period  is  a 
reasonable  expectation.  Distribution  of  AARs  to  the  entire  workforce  raises  the  number 
of  firefighters  impacted  by  each  learning  opportunity. 

4.  Create 

Expanded  fonnal  critiques  and  AARs  create  a  sustainable  homeland  security 
training  and  education  strategy  that  is  based  on  adult  learning  theory.  In  light  of  the  ever¬ 
present  challenge  to  do  more  with  fewer  resources,  exploiting  the  use  of  formal  critiques 
and  AARs  from  conventional  incidents  provides  an  opportunity  to  leverage  resources  in  a 
manner  that  supports  the  simultaneous  accomplishment  of  multiple  objectives.  For 
example,  the  financial  savings  realized  by  exploiting  the  use  of  formal  critiques  and 
AARs  instead  of  conducting  staged  exercises  that  duplicate  practice  of  the  same  activities 
and  tasks  can  be  used  to  pay  for  firefighters  to  attend  technical  training.  The  analysis 
from  this  project  suggests  that  between  nine  and  13  firefighters  can  attend  technical 
training  for  each  formal  critique  that  is  used  in  place  of  a  staged  exercise29’30.  The 
money  saved  can  be  invested  in  providing  training  to  additional  personnel,  creating  a 
mechanism  for  a  fire  department  to  sustain  costly  special  operations  training  programs, 
without  compromising  overall  learning  goal  and  objectives.  Furthennore,  creating  a 
resource  bank  of  archived  AARs  facilitates  learning  at  a  later  date  by  creating  content  for 
company  schools  and  independent  study. 

Formal  critiques  create  opportunities  for  front-line  personnel  to  participate  in 
policy  development.  Collaboration  among  frontline  and  supervisory  personnel  increases 
the  perspectives  from  which  an  issue  is  considered.  The  more  comprehensive  the 
dissection  of  cause  and  effect  relationships,  the  greater  the  probability  that  a  fire 
department  will  identify  which  policies  and  procedures  should  be  memorialized  and  those 
that  need  to  be  changed.  Collaborative  brainstorming  helps  to  create  options  for 

29  For  each  hazardous  materials  FISEEP-compliant  exercise  not  conducted,  the  cost  savings  can  be 
used  to  send  nine — 12  firefighters  to  hazardous  materials  technical  training. 

30  For  each  search  and  rescue  FISEEP-compliant  exercise  that  is  eliminated,  the  money  saved  can  be 
used  to  send  nine — 13  firefighters  to  for  technical  training. 
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strengthening  identified  deficiencies  that  are  realistic  in  tenns  of  implementation  and 
adequacy  for  correcting  the  problem.  Participation  in  policy  development  improves 
stakeholder  buy-in,  which  helps  to  create  an  organizational  culture  that  is  conducive  to 
enacting  policy  changes. 

The  use  of  these  smart  practices  enables  administrators  to  meet  daily  operational 
demands,  while  creating  learning  opportunities  for  effectively  responding  to  incidents  of 
conventional  and  asymmetric  origin.  Additionally,  opportunities  are  created  to  build  the 
collaborative  capacity  necessary  to  critically  assess  and  improve  department  policy  in  a 
comprehensive  manner. 

B.  ACTION  STEPS 

The  desired  value  proposition  for  sustaining  homeland  security  training  and 
education  can  be  achieved  by  expanding  the  scope  of  formal  critiques  and  AARs  from 
conventional  incidents  to  include  “what  if’  questions  about  incidents  of  similar  nature 
with  asymmetric  origins.  To  successfully  assimilate  this  strategy  into  routine  operations, 
it  is  recommended  that  a  fire  department  execute  four  activities: 

•  Commit  to  learning  from  crisis; 

•  Re-evaluate  existing  homeland  security  training  and  education  programs; 

•  Establish  policy  for  conducting  formal  critiques  and  writing  AARs  with  an 
expanded  scope;  and 

•  Periodically  evaluate  response  activities  in  terms  of  the  TCL. 

The  findings  from  this  thesis  project  suggest  that  formal  critiques  and  AARs  as 
mechanisms  for  learning  have  not  been  fully  institutionalized  and  that  valuable  learning 
opportunities  are  possibly  being  missed  (Espedal,  2008;  Holmqvist,  2009;  Carmeli  & 
Sheaffer,  2008).  These  findings  are  consistent  with  Smith  and  Elliott’s  (2007)  assertion 
that  organizations  are  resistant  to  learning  from  crisis.  To  effectively  incorporate  the  use 
formal  critiques  and  AARs  from  conventionally  caused  incidents  as  tools  to  sustain 
homeland  security  training  and  education,  an  organization  must  commit  to  not  only 
learning  for  crisis,  but  to  also  learning  from  crisis.  An  environment  that  encourages 
constructive  criticism  from  all  personnel  ranks  must  be  created,  with  department 
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leadership  willing  to  make  policy  modifications  as  necessary  to  support  the  findings  from 
careful  analysis  of  identified  areas  for  improvement.  A  review  of  disciplinary  procedures 
for  job  performance  deficiencies  may  be  necessary  to  promote  a  critique  environment  that 
is  conducive  to  honest  identification  of  cause  and  effect  relationships.  Cultivation  of 
participant  confidence  that  fonnal  critique  participation  is  purely  for  the  purposes  of 
individual  and  organizational  learning  without  punitive  ramifications  improves  the 
quality  of  the  discussion  and  learning. 

Consistent  dedication  of  the  resources  necessary  to  conduct  formal  critiques  and 
write  AARs  is  necessary,  even  when  this  means  adjusting  the  priority  ranking  of  non¬ 
emergency  organizational  activities  to  accommodate  the  unpredictable  schedule  in  which 
incidents  that  employ  the  activities  and  tasks  contained  in  the  TCL  occur.  The  transition 
of  moving  from  a  culture  that  is  focused  on  learning  for  crisis  to  one  that  includes 
learning  from  crisis  requires  a  commitment  of  time,  energy,  and  resources  to  overcome 
the  challenges  confronted  when  moving  through  the  process  of  change.  However, 
organizations  that  persevere  in  proactively  working  to  learn  from  their  own  experiences 
will  likely  decrease  the  probability  of  experiencing  predictable  surprises  and  lessen  the 
likelihood  of  firefighter  death  or  injury  from  a  cause  that  is  retrospectively  judged  as 
preventable. 

A  department’s  homeland  security  training  and  education  program  should  be  re¬ 
evaluated,  with  the  intent  of  identifying  opportunities  to  adopt  new  strategies  for 
achieving  the  defined  goals  and  objectives.  The  opportunity  for  fire  departments  to 
lessen  the  possibility  of  predictable  surprises  during  homeland  security  incidents  exists, 
but  requires  policy-makers  to  incorporate  different  methods  of  learning  into  existing 
organizational  practices  (Smith  &  Elliott,  2007;  Carmeli  &  Sheaffer,  2008;  Holmqvist, 
2009).  Cross-walking  the  TCL  with  the  agency’s  core  mission,  areas  of  responsibility, 
and  emergency  incident  trends  identifies  potential  opportunities  for  using  expanded 
formal  critiques  and  AARs  from  conventional  incidents  to  support  specialized  training 
and  education  goals  and  objectives.  Comparison  of  the  newly  created  list  with  the 
expected  outcomes  when  one  of  the  traditional  training  and  education  tools  are  employed 
creates  an  awareness  of  possibilities  for  using  new  strategies  to  supplement  the  more 
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conventional  methods.  Adult  learning  theory  principles,  vicarious  learning  opportunities 
created,  execution  frequency,  and  the  breadth  and  variety  of  content  are  factors  to 
consider  when  selecting  tools  for  meeting  learning  objectives.  Similar  to  any  policy 
decision,  a  compromise  on  some  factors  may  be  necessary.  Policy-makers  must  make 
choices  that  best  achieve  the  overall  goals  with  respect  to  the  available  resources. 

Policy  creation  clarifies  the  expectations  for  how  formal  critiques  and  AARs  from 
conventional  incidents  are  used  to  sustain  homeland  security  training  and  education,  as 
well  as  the  roles  and  responsibilities  of  department  members  for  policy  execution. 
Specification  of  the  procedures  for  policy  execution  creates  implementation  uniformity, 
while  helping  to  ensure  that  the  end  product  is  a  useful  learning  tool.  For  example, 
creating  a  formal  critique  facilitator  guide  provides  individuals  responsible  for 
conducting  critiques,  both  those  who  are  new  to  the  facilitation  process  and  those  who  are 
experienced  educators,  with  infonnation  on  how  to  conduct  the  meeting  in  order  to 
optimize  participant  and  organizational  learning.  Similarly,  developing  an  AAR 
document  template  that  details  the  expected  contents  of  each  section  helps  to  ensure  that 
information  is  written  and  organized  in  a  manner  that  promotes  independent  learning.  A 
simple  narrative  of  the  incident  with  an  analysis  of  actions,  recommendations  on  how  to 
strengthen  capabilities,  and  pictures  to  provide  context  for  the  AAR  content  may  be  all 
that  is  required  in  an  AAR  to  facilitate  vicarious  learning.  This  author  has  experimented 
with  developing  AARs  for  conventionally  caused  emergencies  in  Chicago  and  has  found 
such  a  document  structure  to  be  an  effective  format  for  accomplishing  learning 
objectives.  See  the  Appendix  for  an  example.  Policy  and  procedure  development  for 
formal  critiques  and  AARs  as  a  means  to  sustain  homeland  security  training  will  likely 
involve  a  process  of  trial  and  error.  Uniform  implementation  of  each  documented  policy 
version  enables  administrators  to  effectively  assess  which  aspects  of  a  policy  are  effective 
and  those  that  require  further  refinement. 

Periodic  evaluation  of  response  procedures  in  terms  of  the  TCL  is  necessary  to 
ensure  the  on-going  relevance  of  the  findings  from  this  thesis  project  to  a  fire 
department’s  homeland  security  training  and  education  goals  and  objectives.  The  new 
policy  option  is  based  on  the  finding  that  firefighters  execute  the  same  activities  and  tasks 
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for  similar  incident  types,  irrespective  of  whether  the  incident  originated  from  a 
conventional  or  asymmetric  cause.  Therefore,  assessing  the  continued  accuracy  of  this 
assertion  is  needed  in  order  to  remain  confident  in  the  new  strategy  for  training  and 
education.  Routine  re-evaluation  is  recommended,  but  should  minimally  be  done  when 
the  threats  faced  by  and  response  capabilities  of  a  fire  department  change  and  when  there 
are  changes  made  to  federal  capability  recommendations.  Duplication  of  the  process 
used  in  this  thesis  project  to  assess  the  similarities  and  differences  between  the  response 
to  conventional  and  asymmetrically  caused  emergencies  is  likely  sufficient  to  validate 
continued  applicability  of  the  new  learning  strategy.  However,  fire  department 
management  may  wish  to  consider  real-time  evaluation  of  response  procedures  in  tenns 
of  the  activities  and  tasks  for  each  target  capability  since  a  real-time  evaluation  by  on¬ 
scene  chief  officers  will  likely  be  less  time  consuming  than  a  retrospective  analysis.  The 
HSEEP  EEGs  can  still  be  used  for  a  real-time  evaluation. 

In  summary,  incorporating  formal  critiques  and  AARs  from  conventional 
incidents  into  homeland  security  training  and  education  programs  helps  to  decentralize  a 
fire  department’s  strategy  for  learning  and  improves  the  sustainability  of  specialized 
programs.  The  new  value  curve  reduces  resource  requirements  and  the  probability  of 
predictable  surprise.  The  dependence  on  external  resources  for  creating  learning 
opportunities  is  eliminated.  At  the  same  time,  the  redefined  value  curve  raises  the 
number  of  opportunities  for  learning  and  the  number  of  firefighters  impacted  by  each 
opportunity.  A  sustainable  training  and  education  program  that  is  sensitive  to  the 
principles  of  adult  learning  theory  and  provides  opportunities  for  all  stakeholder  levels  to 
engage  in  the  policy  development  process  is  created. 

To  achieve  the  new  value  curve,  engagement  in  four  activities  is  recommended: 
committing  to  the  process  of  learning  from  crisis,  re-evaluating  current  homeland  security 
training  and  education  programs,  establishing  policy  to  support  implementation  of  this 
new  strategy,  and  routinely  evaluating  emergency  response  requirements  in  tenns  of  the 
TCL.  Execution  of  these  recommendations  establishes  an  organizational  culture  in  which 
formal  critiques  and  AARs  from  conventional  incidents  can  be  used  successfully  to 
sustain  homeland  security  training  and  education  programs. 
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VII.  CONCLUSION 


Formal  critiques  and  AARs  from  conventional  emergencies  are  tools  that  can  be 
used  to  sustain  homeland  security  training  and  education.  The  numerous  similarities 
between  the  activities  and  tasks  performed  to  mitigate  emergencies  of  similar  type, 
irrespective  of  whether  the  incident  originated  from  a  conventional  or  asymmetric  cause, 
creates  opportunities  for  firefighters  and  their  organization  to  use  real-world  emergencies 
to  prepare  for  the  homeland  security  challenges  they  may  one  day  face.  The  cost-benefit 
analysis  conducted  in  this  thesis  project  suggests  that  incorporating  the  use  of  formal 
critiques  and  AARs  into  homeland  security  training  and  education  programs  is  an 
efficient  use  of  resources  when  compared  with  the  resource  requirements  for  conducting 
stand-alone,  classroom  based  training  and  Homeland  Security  Exercise  and  Evaluation 
(HSEEP)-compliant  exercises.  Furthermore,  the  analysis  indicates  that  the  proposed 
policy  option  is  more  advantageous  than  traditional  training  and  education  methods 
commonly  used  to  strengthen  homeland  security-related  capabilities  when  process  values, 
robustness,  and  improvability  are  factors  considered  by  policy  makers. 

Integration  of  these  smart  practices  into  routine  department  operations  increases 
the  number  of  opportunities  created  for  firefighters  to  leam,  from  both  their  personnel 
experiences  and  vicariously  through  the  experiences  of  colleagues.  The  phrase  “practice 
makes  perfect”  is  commonly  used  to  describe  the  value  of  routine  performance  of 
essential  skills.  However,  the  phrase  “practice  makes  permanent”  has  greater 
applicability  when  working  to  decrease  the  probability  of  predictable  surprises  through 
training  and  education.  The  more  frequent  the  opportunities  that  firefighters  have  to 
practice  critical  activities  and  tasks,  the  greater  the  probability  of  successful  performance 
when  called  upon  to  execute  these  procedures  to  mitigate  emergencies  of  diverse  origin 
and  complex  nature.  Very  few  emergencies  occur  according  to  a  textbook  script. 
Firefighters  must  be  able  to  adapt  to  the  situation  encountered  and  safely  execute  the 
activities  and  tasks  required  to  protect  the  health,  safety,  and  property  of  those  they  serve. 
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A. 


LIMITATIONS 


A  primary  limitation  of  this  thesis  project  is  the  sample  size  and  the  fact  that  the 
entire  sample  is  taken  from  the  same  data  source.  This  project  uses  four  cases  to  analyze 
two  target  capabilities.  The  cases  selected  are  subjectively  assessed  as  representative  of 
the  types  of  incidents  to  which  the  CFD  responds.  However,  the  small  sample  size 
allows  one  to  question  whether  or  not  there  is  something  unique  about  these  cases  that 
contributes  to  the  findings  that  would  not  be  found  in  the  documentation  from  other 
incidents  of  similar  type.  While  using  data  from  the  same  source  creates  consistency 
among  data  elements  (e.g.,  salary  costs,  fire  company  responsibilities),  it  does  not 
account  for  differences  among  urban  fire  departments  or  assess  how  these  differences 
may  impact  the  findings.  With  Chicago  being  the  third  largest  city  in  the  United  States 
and  uniqueness  of  geography,  demographics,  and  critical  infrastructures,  it  is  possible 
that  the  incidents  to  which  CFD  routinely  responds  may  be  of  unique  complexity  when 
compared  with  other  urban  fire  departments. 

This  thesis  project  relies  entirely  on  historical  documentation.  No  real-time  or 
qualitative  data  is  used.  Real-time  and  qualitative  data  may  provide  a  greater 
understanding  of  decision-making  processes  for  why  an  activity  or  task  was  not 
performed  or  if  it  was  performed  and  just  not  documented. 

Finally,  there  is  no  control  for  variability  among  fire  company  assignments  and 
the  experience  levels  and  documentation  styles  of  each  company  officer  and  chief. 
Documentation  for  each  case  is  aggregated,  but  the  documentation  style  of  each  officer 
and  chief  varied.  It  is  unknown  if  this  is  a  result  of  experience  level,  personal  style,  the 
assignment  that  each  company  was  given  to  perform,  and/or  a  combination  of  multiple 
factors. 

B.  AREAS  FOR  FURTHER  RESEARCH 

To  validate  the  findings  from  this  thesis  project,  application  of  the  research 
methodology  to  cases  obtained  from  a  different  urban  fire  department  is  warranted  to 
determine  if  the  findings  are  replicable.  Future  researchers  should  consider  including 
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real-time  evaluation  of  emergency  response  operations  and  qualitative  interviews  in  the 
research  methodology  to  further  enhance  the  quality  of  the  information  used  for  analysis. 
An  increase  in  the  sample  size  and  including  data  from  more  than  one  fire  department 
will  strengthen  confidence  in  the  reliability  of  findings. 

The  TCL  has  37  capabilities,  of  which  21  are  solely  focused  on  the  response 
mission  area.  Future  researchers  may  wish  to  analyze  whether  or  not  the  activities  and 
tasks  for  additional  capabilities  are  routinely  executed  when  fire  departments  respond  to 
emergencies  of  conventional  origin.  If  such  findings  are  consistent  with  those  of  this 
thesis  project,  the  proposed  policy  option  can  be  expanded  to  include  additional 
capabilities. 

The  response  to  a  homeland  security  incident  requires  the  involvement  of  multiple 
agencies  and  disciplines,  or  a  megacommunity  (Gerencser,  Lee,  Napolitano,  &  Kelly, 
2008).  Some  of  the  individual  target  capabilities  contain  activities  and  tasks  that  reflect 
this  megacommunity  involvement.  Future  researchers  may  consider  analyzing  the 
comprehensive  list  of  target  capabilities  that  are  executed  by  the  megacommunity  in 
response  to  a  conventional  incident,  with  the  intent  of  expanding  the  proposed  policy 
option  beyond  use  by  one  agency.  Incorporating  the  megacommunity  into  the  fonnal 
critique  and  after  action  process  may  provide  valuable  opportunities  for  the  entire  group 
to  prepare  for  homeland  security  incidents  by  optimizing  learning  opportunities  from 
conventionally-caused  incidents. 
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APPENDIX:  SAMPLE  AFTER  ACTION  REPORT 


Trench  Rescue 
After  Action  Review 

Incident  Date:  May  27,  2009 
Incident  Location:  5900  W.  Belmont  Ave 
After  Action  Date:  June  2,  2009 

Incident  Overview 

At  1225,  a  member  of  a  construction  company  called  911  to  report  that  one  of  their 
employees  was  buried  up  to  his  knees  in  a  below  grade  hole  secondary  to  the  earth  caving 
in.  Upon  arrival  at  the  scene,  Engine  94  and  the  Battalion  Chief  confirmed  that  one 
person  was  trapped  in  an  unsupported  trench  and  that  the  trench  was  approximately  five 
feet  by  five  feet  and  ten  feet  deep.  The  victim  reported  that  his  legs  were  pinned 
underneath  a  pipe  and  that  they  hurt.  The  Chicago  Fire  Department  (CFD)  initiated  a 
trench  rescue  response.  Thirty-one  companies  responded  to  the  scene  and  mitigation  of 
the  incident  took  approximately  five  hours. 

Analysis  of  Actions 

1 .  The  first  on-scene  companies  appropriately  used  the  plywood  that  was  available 
at  the  construction  site  to  brace  the  hole,  provided  the  victim  with  oxygen,  tied  a 
rope  around  the  victim’s  body,  gave  the  victim  a  shovel,  and  called  for  a  full 
trench  rescue  response.  The  companies  did  not  try  to  pull  the  victim  from  the 
hole.  The  companies  followed  departmental  procedure  and  correctly 
implemented  basic  bracing  techniques  learned  through  training. 

2.  After  using  plywood  to  further  brace  the  hole,  squad  company  members  assessed 
the  situation  further  and  learned  that  the  most  unstable  area  was  the  concrete  on 
the  north  side  of  the  hole,  that  a  drain  tile  had  broken  and  washed  pea  gravel  into 
the  trench,  that  there  was  a  catch  basin  in  the  hole,  and  that  the  victim’s  legs  were 
in  fact  bent  underneath  him  and  caught  in  a  pipe.  Members  placed  struts  to 
further  stabilize  the  hole,  with  a  priority  of  protecting  the  victims  head  and  chest. 
The  air  bags  and  plywood  placed  were  successful  in  stopping  the  influx  of  pea 
gravel.  Operations  were  stopped  at  frequent  intervals  so  that  supervisory 
personnel  could  ensure  that  the  shoring  was  correct  and  that  the  appropriate  safety 
measures  were  in  place.  Members  followed  established  departmental  protocols 
for  trench  operations. 

3.  Once  the  hole  was  stabilized,  squad  members  rotated  going  into  the  hole  to  dig 
the  debris  away  from  the  victim.  At  all  times,  two  members  were  harnessed  and 
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in  position  to  initiate  secondary  rescue  operations,  if  needed.  Members  followed 
established  departmental  safety  standards  for  trench  operations. 

4.  Team  companies  set-up  the  cutting  station,  brought  struts  and  the  paratech  system 
for  inflating  the  air  bags  to  the  hole,  kept  the  area  of  the  collapse  clear  of 
pedestrian  traffic,  and  provided  generalized  incident  support.  The  precision  of 
measurements  and  wood  cutting  greatly  decreased  the  time  required  for  incident 
mitigation. 

5.  En-route,  the  Fire  Alarm  Office  provided  responding  companies  with  a  status 
update  via  the  Mobile  Data  Computer.  Receipt  of  this  information  prior  to  arrival 
on-scene  helped  responding  companies  pre-plan  and  prioritize  the  actions  they 
would  taken  upon  arrival. 

6.  EMS  personnel  started  an  IV  and  administered  IV  fluids,  pain  medications,  and 
other  medications  as  appropriate  for  a  victims  with  crush  injuries.  Paramedics 
routinely  monitored  the  victim’s  blood  sugar  and  calculated  rates  of  fluid 
replacement.  EMS  considered  requesting  a  helicopter  for  transport;  however,  the 
amount  of  debris  on-scene  would  have  created  safety  hazards.  EMS  provided 
excellent  care  to  maintain  patient  stabilization  through  the  entire  incident  and 
subsequent  transport. 

7.  The  Water  Department’s  vacuum  truck  was  brought  to  the  scene  to  help  remove 
the  earth  faster.  The  truck’s  hose  is  8”  in  diameter.  Due  to  the  size  of  the  hole 
and  the  on-going  caving  of  the  surrounding  earth,  a  smaller  diameter  nozzle  was 
needed  to  better  control  the  rate  of  removal.  CFD  and  Water  Department 
personnel  worked  together  to  create  a  makeshift  nozzle  of  smaller  diameter  out  of 
duct  tape,  a  five-gallon  bucket,  and  an  orange  traffic  cone.  The  new  nozzle 
worked  to  control  the  sucking  and  facilitate  removal  of  the  earth  trapping  the 
victim.  CFD  has  since  received  the  smaller  diameter  nozzles  that  were  ordered 
prior  to  the  incident. 

8.  Fire  Ground  was  used  for  on-scene  communications;  however,  a  runner  was  used 
for  relaying  requests  between  the  hole  and  the  cutting  station.  A  dedicated 
channel  for  communication  between  members  operating  at  the  hole  and  the 
cutting  station  is  included  in  the  general  order  on  trench  rescue  operations  that  is 
being  written. 

9.  While  executing  primary  rescue  operations,  officers  were  actively  developing 
contingency  plans.  If  primary  operations  had  failed  and  the  victim  was  in  fact 
found  to  be  pinned  behind  the  catch  basin,  companies  had  planned  to  dig  through 
the  catch  basin  to  perform  the  extraction.  This  operation  would  have  been 
complicated  by  the  fact  that  there  was  a  distance  of  20’  between  the  top  and 
bottom  of  the  catch  basin.  Officers  displayed  critical  thinking  in  developing 
contingency  plans. 
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10.  Squad  members  prepared  an  A-frame  to  lift  the  victim  from  the  hole,  however, 
the  victim  climbed  up  the  ladder  himself  and  the  A-frame  was  not  needed. 
Preparation  of  this  device  in  anticipation  of  possible  need  was  appropriate. 

11.  Tape  was  put  up  around  the  perimeter  and  lines  were  painted  on  the  ground  to 
identify  restricted  areas,  however,  these  measures  were  not  independently 
sufficient  to  restrict  access  to  the  scene. 

12.  A  responder  rehab  area  was  established  and  utilized.  Officers  did  a  good  job  of 
ensuring  that  members  were  given  sufficient  periods  for  rest  and  rehab. 

Recommendations/Improvement  Plan 

1 .  Include  the  vacuum  truck  in  the  general  order  that  is  being  written  as  a  resource  to 
be  considered  requesting. 

2.  Continue  to  include  training  on  use  of  the  vacuum  truck  during  trench  rescue 
training  classes.  Consider  joint  training  with  personnel  from  the  Department  of 
Water  Management  on  using  the  truck. 

3.  Incorporate  round  trenches  in  training  classes.  Classes  typically  focus  on 
rectangular  trenches.  Inserting  struts  to  brace  a  round  trench  creates  unique 
challenges. 

4.  Continue  to  send  members  of  special  operations  and  team  companies  to  trench 
and  confined  space  classes. 

5.  Assign  personnel  to  restrict  pedestrian  traffic  to  unstable  or  dangerous  areas 
within  the  scene.  Response  personnel  who  are  not  providing  direct  service  or 
support  should  not  access  restricted  areas. 

Impact  of  Training 

AAR  participants  were  asked  whether  or  not  participation  in  special  operations  training 
classes  has  had  an  impact  on  the  way  response  operations  for  trench  rescue  are  conducted 
and,  if  so,  to  describe  how. 

1.  A  Lieutenant  with  23  years  of  experience  said,  “Trench  operations  training  has 
helped  to  minimize  the  risk  to  fire  fighters.  Previously,  we  jumped  right  in  the 
hole.  Now  we  stabilize  the  hole  first.” 

2.  A  District  Chief  with  29  years  of  experience  said,  “Training  has  really  helped  to 
facilitate  companies  working  together.  The  impact  of  cross  company  training  and 
drilling  was  evident  during  the  entire  incident.  The  technical  knowledge  of 
responding  companies  on  trench  operations  was  excellent.” 
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3.  A  company  officer  said,  “Having  all  officers  on  the  same  page,  using  universal 
language,  and  having  an  understanding  of  what  was  going  on  and  why  really 
helped  to  facilitate  operations.” 

4.  An  officer  of  the  squad  said,  “We  knew  that  we  had  to  have  a  Plan  B  and  Plan  C 
incase  our  initial  strategy  did  not  work.  We  started  working  on  the  contingency 
plans  right  away.” 

5.  A  firefighter  said,  “Everyone  knew  what  was  being  requested.  We  did  not  waste 
time  trying  to  figure  out  what  the  different  pieces  of  equipment  are.” 

6.  The  Assistant  Deputy  Fire  Commissioner  for  Special  Operations  said,  “Trench 
operations  normally  take  8-10  hours.  This  one  took  5  hours.  The  teamwork  and 
efficient  manner  in  which  operations  were  conducted  really  helped  to  spare  the 
victim  further  injury.” 

7.  A  squad  officer  said,  “Skills  learned  during  collapse,  trench,  rope,  and  confined 
space  classes  were  used  in  this  response.” 

8.  An  investigator  from  the  Occupational  Safety  and  Health  Administration  (OSHA) 
complimented  CFD  on  adhering  to  safety  standards,  especially  wearing  harnesses 
when  working  around  the  hole. 

Summary 

In  many  ways,  this  trench  rescue  operation  was  a  success.  The  victim  was  transported  to 
the  hospital  in  stable  condition  and  discharged  from  the  hospital  the  next  day. 
Throughout  the  rescue  operation,  teamwork  among  Fire  Department  personnel  prevailed 
and  Departmental  safety  standards  were  maintained.  This  incident  demonstrates  the 
value  of  the  financial  investment  that  has  been  made  in  training  personnel  to  perform 
rescue  operations  in  a  safe  and  effective  manner.  The  victim  was  successfully  rescued 
and  there  were  no  injuries  to  any  on-scene  rescue  personnel. 
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